Image pseudo-analysis category incremental learning method and device based on sampling points

Abstract

The present invention discloses a sampling point-based incremental learning method and device for pseudo-analytic category of images. The method comprises the following steps: firstly, an image classification network is constructed, and then a basic data set is used to pre-train a background network. After the pre-training is completed, the background network is frozen and a one-dimensional feature is obtained. Then, an analytic classifier is connected and a virtual node is constructed. Subsequently, the one-dimensional feature is randomly dimensionally expanded and input into the analytic classifier for realignment. After the completion, a scaling factor is calculated and an initial sampling point set is obtained. Finally, an incremental learning phase is entered. Each incremental learning phase requires mixing the initial sampling point set with the incremental data set of the current phase. The mixed data set is used to train the analytic classifier and the sampling point set of the current phase is obtained as the initial sampling point set of the next phase. The incremental data set of the next phase is obtained and repeated incremental learning is performed. The method can alleviate the problem of catastrophic forgetting, has strong privacy and high precision, and can realize incremental learning of analytic categories of images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and incremental learning, and in particular relates to a method and device for incremental learning of image pseudo-analysis categories based on sampling points. Background Art

[0002] The current mainstream machine learning models have achieved remarkable results in dealing with specific tasks, but the basis of their success often relies on human experience and knowledge of specific problems. However, these models often seem to be unable to cope with complex and changeable tasks in the real world. Traditional machine learning models are usually trained on fixed data sets to solve single tasks; but since the data in real applications is continuously flowing and constantly changing, the model needs to cope with new data and tasks. However, once deployed, traditional machine learning models are difficult to adapt to new environments and data, which limits their flexibility and effectiveness in practical applications. In order to make up for this deficiency, incremental learning came into being. Its goal is to enable the model to continuously learn and optimize in a constantly changing data environment to ensure that it still performs well when facing new data. The development of incremental learning can be traced back to the early research in the field of neural networks and machine learning. Early research mainly focused on how to improve the learning efficiency and performance of the model through incremental training methods; with the development of deep learning technology, incremental learning has gradually become an independent and important research direction. It is not only an important research direction in the field of machine learning, but also an important path to achieve higher-level artificial intelligence. It is expected to bring revolutionary progress in the fields of unmanned driving, intelligent robots, medical diagnosis, etc.

[0003] The core concept of incremental learning is to imitate the learning process of the human brain, and gradually improve the problem-solving ability by continuously accumulating and integrating new knowledge. An important feature of incremental learning is that it can retain the learned knowledge when dealing with new tasks, which is in sharp contrast to traditional machine learning methods. Traditional machine learning methods often suffer from catastrophic forgetting when introducing new tasks, that is, the model loses the previously learned knowledge when learning new tasks, especially in networks using back-propagation learning. Incremental learning strives to overcome this problem by optimizing algorithms and model structures, and realizes the effective integration of new and old knowledge. Early methods to alleviate catastrophic forgetting are usually to store previous data and periodically replay sample interleaved data extracted from old samples and new samples (ShinH, Lee JK, Kim J, et al. Continual Learning with Deep Generative Replay[J]. 2017. DOI: 10.48550 / arXiv.1705.08690.). However, the disadvantage is that old information needs to be explicitly stored, resulting in a large demand for working memory; in addition, for a fixed number of neural resources, a special mechanism needs to be designed to protect the comprehensive knowledge from being overwritten by the learning of new information. In addition, catastrophic forgetting can also be alleviated by allocating additional neural resources when needed, but this approach may lead to scalability issues, significantly increasing the computational workload of the neural network and turning it into a very large architecture. In the prior art, researchers have proposed a variety of incremental learning methods, including replay-based methods, regularization-based methods, and parameter isolation-based methods. These methods have made some progress in solving the problem of catastrophic forgetting, but there are still many challenges and limitations.

[0004] For replay-based incremental learning methods (such as "Parisi GI, Kemker R, Part JL, et al. Continual lifelong learning with neural networks: A review [J]. Neural Networks: The Official Journal of the International Neural Network Society, 2019 (113-): 113."): This method replays the stored old task data during incremental learning to overcome catastrophic forgetting, such as memory replay. This method has achieved good results, but because it needs to store old data, it leads to large memory usage, and the stored old data has the security issue of leaking user privacy. For incremental learning methods based on regularization (such as "French R M. Catastrophic forgetting in connectionist networks [J]. Trends in Cognitive Sciences, 1999. DOI: 10.1016 / S1364-6613 (99) 01294-2."): Regularization methods that add additional constraints to the loss function. This method prevents weights that are more important to old knowledge from changing excessively during the process of learning new data, thereby overcoming catastrophic forgetting to a certain extent, such as LwF, EWC and other methods. In addition, this method does not store old data and has strong security. However, this method needs to prevent the excessive deviation of important parameters of old tasks in the training of new tasks. Therefore, how to quantitatively judge the importance of weights is a problem. For incremental learning methods based on parameter isolation (such as "Ratcliff R. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. [J]. Psychological Review, 1990, 97 (2): 285-308. DOI: 10.1037 / / 0033-295X.97.2.285."): This method flexibly configures model parameters in the incremental learning process according to needs by freezing / unfreezing, adding / deleting model parameters, etc. to achieve the effect of overcoming catastrophic forgetting. However, this method is more dependent on the performance of the generator and has poor applicability.

[0005] In recent years, some scholars have proposed the Analytical Class-Incremental Learning (ACIL) method, which uses the properties of the least squares method of multivariate linear functions to calculate the analytical solution of incremental learning through the recursive formula of the matrix, thereby realizing category incremental learning. It not only ensures privacy, but also achieves surprisingly good performance. In addition, Analytical Class-Incremental Learning also has good theoretical interpretability. However, this method only performs incremental learning on the last linear layer of the model, which obviously limits the performance of this type of method; and as the incremental learning continues, the complexity increases, causing the error accumulation problem to become significant; at the same time, this method is only applicable to incremental learning of linear models and cannot be extended to nonlinear models, further limiting its applicability. Summary of the invention

[0006] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a method and device for incremental learning of pseudo-analytic categories of images based on sampling points. On the basis of the incremental learning method of analytic categories, the sampling point method is used for improvement. It can not only process linear models, but also effectively handle the complexity of nonlinear models during the training process, and maintain the stability and accuracy of the model at different training stages.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] The first object of the present invention is to provide an image pseudo-analysis category incremental learning method based on sampling points, comprising the following steps:

[0009] S1. Construct an image classification network, including a background network and a parsing classifier; the background network extracts image features through a convolutional neural network, and the parsing classifier uses a fully connected network to classify image features and outputs classification results in a one-hot encoding form; the parsing classifier is constructed based on a linear model or a nonlinear model;

[0010] S2. Use the reverse gradient descent algorithm to pre-train the background network followed by the softmax classifier on the basic data set. After the training, freeze the parameters of the background network and obtain the one-dimensional features of the basic data set.

[0011] S3, remove the softmax classifier, connect the parsing classifier after the background network with frozen parameters, and add a virtual node in the parsing classifier to participate in subsequent operations; the corresponding classification result adds a dimension, which is the virtual label of the corresponding virtual node that is always 0;

[0012] S4, performing random dimensional expansion on the one-dimensional features of the basic data set, inputting the features into the analytical classifier and realigning them using the reverse gradient descent algorithm, and calculating the scaling coefficient after completion to obtain an initial sampling point set; the number of sampling points in the initial sampling point set is determined by the analytical classifier and kept constant;

[0013] S5, obtaining the incremental data set of the current stage, inputting the background network of the frozen parameters to obtain the newly added feature set, and mixing it with the initial sampling point set to obtain a mixed data set;

[0014] S6, using the MSE loss function to perform incremental learning on the parsing classifier of the previous stage on the mixed data set through the reverse gradient descent algorithm, and updating the sampling points as the initial sampling point set of the next stage;

[0015] S7, obtaining the incremental data set of the next stage, repeating steps S5-S6 until the set stage is reached or there is no incremental data set of the next stage, and obtaining a learned image classifier.

[0016] As a preferred technical solution, in step S2, the pre-training process of the background network followed by a softmax classifier is:

[0017] The basic data set is input into the background network for feature extraction to obtain the basic data feature set;

[0018] Flatten the basic data feature set in one dimension to obtain the one-dimensional features of the basic data set;

[0019] The subsequent softmax classifier classifies the one-dimensional features of the basic data set and outputs the classification results in the form of one-hot encoding, expressed as: Z = f softmax (f flat (f CNN (X0,W CNN ),W FCN )); where f softmax is the softmax classifier, f flat is a one-dimensional flattening operation, f CNN is the background network, X0 is the basic data set, W CNN is the background network parameter, W FCN is the parameter of the linear output layer in the one-dimensional flattening operation, which adjusts the dimension of the one-dimensional feature to the same dimension as the true label of the basic dataset;

[0020] The loss function is calculated based on the classification results and the true labels of the basic data set, and the network parameters are updated through the reverse gradient descent algorithm until the loss function converges or the maximum number of iterations is reached, freezing the parameters of the background network.

[0021] As a preferred technical solution, in step S4, the input parsing classifier uses a reverse gradient descent algorithm for realignment, the steps are:

[0022] Use the activation function to randomly expand the one-dimensional features of the basic data set to obtain the expanded feature set

[0023]

[0024] Among them, f act is the activation function, f flat is a one-dimensional flattening operation, f CNN is the background network, X0 is the basic data set, W CNN is the background network parameter, W FCN is the parameter of the linear output layer in the one-dimensional flattening operation, which adjusts the dimension of the one-dimensional feature to the same dimension as the true label of the basic dataset, W FE Expand the matrix for randomly generated parameters;

[0025] The expanded feature set is input into the parsing classifier and realigned using the MSE loss function through the reverse gradient descent algorithm; during the realignment process, the optimization function of the parsing classifier parameters is:

[0026]

[0027] Among them, W IL To analyze the parameters of the classifier, Y0 is the true label set of the basic data set, f IL To parse the classifier, represents the predicted label set of the basic dataset, ||·|| F is the Frobenius norm;

[0028] After completion, a set number of sampling points and their corresponding prediction labels are randomly generated, the sampling points and their corresponding prediction labels are optimized based on the expanded feature set and prediction label set, the scaling coefficient is calculated and the initial sampling point set is obtained.

[0029] As a preferred technical solution, the steps for obtaining the initial sampling point set are:

[0030] A set number of sampling points and their corresponding prediction labels are randomly generated; the coordinates of the sampling points and their corresponding prediction labels are randomly generated;

[0031] Based on the set number of sampling points and the number of basic data sets, the scaling factor λ0 is calculated as follows:

[0032]

[0033] Among them, nsam is the set number of sampling points, n0 is the number of basic data sets;

[0034] Calculate the first-order gradient vector g0 and second-order gradient matrix G0 of the MSE loss function on the basic data set for the parameters of the trained analytic classifier, as well as the first-order gradient vector of the MSE loss function on the sampling point for the parameters of the trained analytic classifier. and the second-order gradient matrix

[0035] Use the gradient descent algorithm to optimize the sampling point coordinates and their corresponding prediction labels respectively, so that the second-order gradient matrix is approximately equal to λ0G0 and the first-order gradient vector Approximately equal to λ0g0, the initial sampling point set is obtained.

[0036] As a preferred technical solution, in step S6, the incremental learning of the parsing classifier in the previous stage on the mixed data set is specifically as follows:

[0037] Based on the number of newly added labels in the incremental data set at the current stage, the constructed virtual nodes and their corresponding parsing classifier parameters at the current stage are copied, and the last copied virtual node is used as a new virtual node, and the remaining copied virtual nodes are used as real nodes and the real nodes of the parsing classifier at the current stage to participate in subsequent incremental learning; the number of copies is equal to the number of newly added labels; the virtual label corresponding to the new virtual node is always 0;

[0038] Train the parsing classifier of the previous stage on the mixed data set to obtain the parsing classifier trained in the current stage;

[0039] Regenerate a set number of sampling points and their corresponding prediction labels; calculate the scaling factor for the current stage based on the set number of sampling points and the amount of data accumulated by the parsing classifier in history;

[0040] The first-order gradient vector and second-order gradient matrix of the MSE loss function for the parameters of the analytical classifier trained in the current stage are calculated on the mixed data set of the current stage and the regenerated sampling points, and the gradient alignment and error correction of the regenerated sampling points are performed as the initial sampling point set for the next stage.

[0041] As a preferred technical solution, during the training process, the optimization function of the analytical classifier parameters is:

[0042]

[0043] Among them, W IL To parse the classifier parameters, Y k is the true label set of the current k-th stage incremental dataset, is the newly added feature set of the current k-th stage incremental data set, f IL To parse the classifier, represents the predicted label set of the current k-th stage incremental dataset, ||·|| F is the Frobenius norm, λ k-1 is the scaling factor of the initial sampling point set, is the weight of the loss function of the initial sampling point set, is the initial sampling point set, is the initial sampling point set The true label set, is the initial sampling point set The predicted label set of

[0044] The calculation formula of the scaling factor is:

[0045]

[0046] Among them, λ k is the scaling factor of the current kth stage, n sam is the set number of sampling points, n k The amount of data accumulated to parse the classifier history.

[0047] As a preferred technical solution, the regenerated sampling points are subjected to gradient alignment and error correction as the initial sampling point set for the next stage, specifically:

[0048] Calculate the first-order gradient vector g of the MSE loss function on the mixed data set of the current stage for the parameters of the analytical classifier trained in the current stage k and the second-order gradient matrix G k ;

[0049] Calculate the first-order gradient vector of the MSE loss function on the sampling points regenerated in the current stage for the parameters of the analytical classifier trained in the current stage and the second-order gradient matrix

[0050] Use the gradient descent algorithm to optimize the sampling point coordinates so that the second-order gradient matrix Approximately equal to λ k G k Complete gradient alignment;

[0051] The optimized sampling points are input into the analytical classifier trained in the current stage to obtain the predicted labels of the regenerated sampling points, and the gradient descent algorithm is used to optimize the predicted label coordinates of the sampling points so that the first-order gradient vector Approximately equal to λ k g k Complete error correction;

[0052] Repeat the above steps until all regenerated sampling points are optimized, and the optimized sampling point set is used as the initial sampling point set for the next stage.

[0053] The second object of the present invention is to provide an image pseudo-analysis category incremental learning system based on sampling points, including a classification network construction module, a background network pre-training module, a virtual node addition module, an analysis classifier pre-training module, a data mixing module, an analysis classifier training module and an incremental learning termination module;

[0054] The classification network construction module is used to construct an image classification network, including a background network and a parsing classifier; the background network extracts image features through a convolutional neural network, and the parsing classifier uses a fully connected network to classify image features and outputs classification results in a one-hot encoding form; the parsing classifier is constructed based on a linear model or a nonlinear model;

[0055] The background network pre-training module is used to pre-train the background network followed by a softmax classifier on the basic data set using a reverse gradient descent algorithm, and after completion, the parameters of the background network are frozen and one-dimensional features of the basic data set are obtained;

[0056] The virtual node adding module is used to remove the softmax classifier, connect the parsing classifier after the background network with frozen parameters, and add a virtual node in the parsing classifier to participate in subsequent operations; the corresponding classification result adds a dimension, which is the virtual label of the corresponding virtual node that is always 0;

[0057] The analytical classifier pre-training module is used to perform random dimensional expansion on the one-dimensional features of the basic data set, input the analytical classifier to perform realignment using the reverse gradient descent algorithm, and after completion, calculate the scaling coefficient and obtain an initial sampling point set; the number of sampling points in the initial sampling point set is determined by the analytical classifier and kept constant;

[0058] The data mixing module is used to obtain the incremental data set of the current stage, input the background network of the frozen parameters to obtain the newly added feature set, and mix it with the initial sampling point set to obtain a mixed feature set;

[0059] The parsing classifier training module is used to perform incremental learning on the parsing classifier of the previous stage on the mixed feature set by using the MSE loss function through the reverse gradient descent algorithm, and update the sampling points as the initial sampling point set of the next stage;

[0060] The incremental learning termination module is used to obtain the incremental data set of the next stage, repeatedly execute the data mixing module and the analytical classifier training module until the set stage is reached or there is no incremental data set of the next stage, and obtain a learned image classifier.

[0061] A third object of the present invention is to provide an electronic device, comprising:

[0062] at least one processor; and a memory in communication with the at least one processor; wherein,

[0063] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned sampling point-based image pseudo-parse category incremental learning method.

[0064] The fourth object of the present invention is to provide a computer-readable storage medium storing a program, which, when executed by a processor, implements the above-mentioned sampling point-based image pseudo-analysis category incremental learning method.

[0065] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0066] The method of the present application uses a sampling point method to improve the incremental learning method of the parsing category, and freezes the parameters of the background network by pre-training, thereby reducing the amount of calculation of the background network participating in subsequent training; then gradient realignment is performed to construct an initial sampling point set, retaining the knowledge learned by the parsing classifier on the basic data set; then incremental learning is started, and the sampling points are updated to retain the knowledge after each stage of incremental learning; compared with other incremental learning methods, the initial sampling points of the present application store the knowledge of the old training data in a centralized manner, so that subsequent incremental learning can be completed without accessing the old training data, and has good privacy and security; at the same time, the stored sampling points cannot restore any training data through any reverse engineering, but only meet the mathematical properties of incremental learning, which can greatly reduce catastrophic forgetting; and the learned knowledge is preserved by the sampling point method, with an accuracy close to the parsing level and can be generally applied to nonlinear models. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 It is a flow chart of the incremental learning method of image pseudo-analysis category based on sampling points in an embodiment of the present invention.

[0069] Figure 2 Schematic diagram of virtual node and virtual label expansion in an embodiment of the present invention.

[0070] Figure 3 Schematic diagram of second-order gradient calculation of output components in different network structures in an embodiment of the present invention.

[0071] Figure 4 This is a graph showing changes in accuracy during incremental learning of a parsing classifier in an embodiment of the present invention.

[0072] Figure 5 This is a graph showing changes in the first-order gradient error during incremental learning of the analytical classifier in an embodiment of the present invention.

[0073] Figure 6 This is a graph showing changes in the second-order gradient error during incremental learning of the analytical classifier in an embodiment of the present invention.

[0074] Figure 7 4 is an overall block diagram of an image pseudo-analysis category incremental learning system based on sampling points in an embodiment of the present invention.

[0075] Figure 8 FIG. 4 is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0077] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0078] The analytical learning method was originally proposed to overcome the problems of gradient explosion / gradient vanishing and too many iterations in the traditional BP neural network training. This method is based on the least squares method of linear models and requires the calculation of the matrix pseudo-inverse to obtain the result, so it is also called the pseudo-inverse learning method. The analytical category incremental learning ACIL method is an incremental learning method based on the analytical learning idea. This method does not perform training during the incremental learning process, but obtains the analytical results of incremental learning through algebraic operations between matrices. It has the advantages of strong privacy, high theoretical interpretability, and high accuracy; but there are problems such as error accumulation and inapplicability to nonlinear models. Based on the ACIL method, this application introduces the sampling point method to improve the ACIL method, thereby realizing incremental learning of image analytical categories, thereby achieving the purpose of image classification.

[0079] like Figure 1 As shown, the image pseudo-analysis category incremental learning method based on sampling points in this embodiment includes the following steps:

[0080] S1. Build an image classification network, including a background network and a parsing classifier;

[0081] Like the ACIL method, the image classification network of the present application includes a background network and a parsing classifier, wherein the background network extracts image features through a convolutional neural network (CNN), and the parsing classifier uses a fully connected network (FCN) to classify image features and outputs classification results in the form of one-hot encoding; the parsing classifier can be a linear model or a nonlinear model;

[0082] S2. Use the reverse gradient descent algorithm to pre-train the background network followed by the softmax classifier on the basic data set. After the pre-training, freeze the parameters of the background network to obtain the one-dimensional features of the basic data set.

[0083] Furthermore, the present application first pre-trains the background network and then freezes its parameters, specifically:

[0084] S201, inputting the basic data set into the background network to extract features, thereby obtaining a basic data feature set;

[0085] S202, flattening the basic data feature set in one dimension to obtain one-dimensional features of the basic data set;

[0086] S203, the subsequent softmax classifier classifies the one-dimensional features of the basic data set and outputs the classification results in the form of one-hot encoding, expressed as: Z = f softmax (f flat (f CNN (X0,W CNN ),W FCN)); where f softmax is the softmax classifier, f flat is a one-dimensional flattening operation, f CNN is the background network, X0 is the basic data set, W CNN is the background network parameter, W FCN is the parameter of the linear output layer in the one-dimensional flattening operation, which adjusts the dimension of the one-dimensional feature to the same dimension as the true label of the basic dataset;

[0087] S204. Calculate the loss function based on the classification result and the real label set of the basic data set, and update the network parameters through the backpropagation (BP) algorithm (such as stochastic gradient descent, SGD) until the loss function converges or reaches the maximum number of iterations, and freeze the parameters of the background network to prevent it from participating in subsequent incremental learning training.

[0088] It should also be noted that the softmax classifier is only used to pre-train the background network and is removed after completion and does not participate in subsequent incremental learning.

[0089] S3. Remove the softmax classifier, connect the parsing classifier after the background network with frozen parameters, and add a virtual node in the parsing classifier to participate in subsequent operations; correspondingly, the classification result of the parsing classifier adds a dimension, which is the virtual label of the corresponding virtual node that is always 0.

[0090] S4. Perform random dimensional expansion on the one-dimensional features of the basic data set and input them into the parsing classifier for realignment using the reverse gradient descent algorithm. After completion, calculate the scaling factor and obtain the initial sampling point set, where the number of sampling points is determined by the parsing classifier and remains constant.

[0091] Furthermore, the one-dimensional features obtained by pre-training the background network are expanded in dimension (Feature expansion process), and the parsing classifier is re-aligned (Re-alignment), specifically:

[0092] S401. Use the activation function to randomly expand the one-dimensional features of the basic data set to obtain the expanded feature set.

[0093]

[0094] Among them, f act is the activation function, f flat is a one-dimensional flattening operation, f CNN is the background network, X0 is the basic data set, W CNNis the background network parameter, W FCN is the parameter of the linear output layer in the one-dimensional flattening operation, which adjusts the dimension of the one-dimensional feature to the same dimension as the true label of the basic dataset, W FE is a randomly generated parameter expansion matrix; in this embodiment, the activation function f act Use ReLU activation function;

[0095] S402, input the expanded feature set into the parsing classifier and use the MSE loss function to perform realignment through the reverse gradient descent algorithm; during the realignment process, use f IL represents the parsing classifier, W IL Represents the parameters of the parsing classifier, then the optimization function of the parsing classifier parameters is:

[0096]

[0097] Among them, Y0 is the true label set corresponding to the basic data set, which is also represented in the form of one-hot encoding. represents the predicted label set of the basic dataset, ||·|| F is the Frobenius norm;

[0098] S403: After completion, a set number of sampling points and their corresponding prediction labels are randomly generated, the sampling points and their corresponding prediction labels are optimized based on the expanded feature set and prediction label set, the scaling coefficient is calculated and the initial sampling point set is obtained.

[0099] Specifically, f is trained through several rounds of back-propagation gradient descent IL , get the initial training results; in order to allow incremental learning to proceed, IL After training is completed, an initial sampling point set needs to be given, and the steps to obtain it are:

[0100] Randomly generate a set number of sampling points and their corresponding predicted labels; the number of sampling points is determined by the designed f IL The coordinates of the sampling points and their corresponding predicted labels are randomly generated.

[0101] When the gradient descent algorithm is actually used to train the model, there is a problem of gradient error. For this reason, this application calculates the scaling factor λ0 based on the set number of sampling points and the number of basic data sets to correct the error of the first-order gradient. At the same time, it can also correct the error of some elements in the second-order gradient. The formula is:

[0102]

[0103] Among them, n samis the set number of sampling points and does not change with the incremental learning process. n0 is the amount of data accumulated historically by the parsing classifier. Here, since the basic data set is used for realignment first, it is the number of basic data sets. As incremental learning continues, the amount of data accumulated historically by the parsing classifier will continue to increase.

[0104] Next, we calculate the first-order gradient vector g0 and the second-order gradient matrix G0 of the MSE loss function on the basic data set for the parameters of the trained analytical classifier, as well as the first-order gradient vector of the MSE loss function on the sampling point for the parameters of the trained analytical classifier. and the second-order gradient matrix It should be noted that the second-order gradient vector is a Hessian matrix, and those skilled in the art are aware of its meaning and related calculation process, which will not be elaborated in this embodiment.

[0105] Use the gradient descent algorithm to optimize the sampling point coordinates and their corresponding prediction labels respectively, so that the second-order gradient matrix is approximately equal to λ0G0 and the first-order gradient vector Approximately equal to λ0g0, the initial sampling point set is obtained. In this embodiment, when the error is less than or equal to 2.5%, it is considered to be approximately equal.

[0106] S5, obtain the incremental data set of the current stage, input the background network with frozen parameters to obtain the newly added feature set, and mix it with the initial sampling point set to obtain a mixed data set; since the initial sampling point set records the features and predicted labels learned in the previous stage, it is not necessary to extract features through the background network in the current stage and can be mixed with the current newly added feature set, reducing the amount of calculation. It can be understood that the realignment process in this embodiment can be used as the initial stage (Phase 0) in the incremental learning process.

[0107] S6. Use the MSE loss function to perform incremental learning (Class incremental learning) on ​​the parsing classifier of the previous stage on the mixed data set through the reverse gradient descent algorithm, and update the sampling points as the initial sampling point set of the next stage.

[0108] Furthermore, the mixed data set is used to train the parsing classifier in the previous stage, specifically:

[0109] S601. Based on the number of new real labels in the incremental data set of the current stage, the constructed virtual nodes and their corresponding parsing classifier parameters in the current stage are copied, and the last copied virtual node is used as a new virtual node. The remaining copied virtual nodes are used as real nodes and the real nodes of the parsing classifier of the current stage to participate in subsequent incremental learning; wherein the number of copies is equal to the number of new labels; the virtual label corresponding to the new virtual node is always 0.

[0110] like Figure 2 As shown in the figure, if the number of new labels in the incremental data set in the current k-th stage of incremental learning is N k , then you need to copy the virtual node and its corresponding parsing classifier parameters at the current stage to N k times. In this case, the output dimension of the parsing classifier increases by N k The last copied virtual node is used as the new virtual node, and the remaining copied virtual nodes become additional real nodes of the parsing classifier, and participate in subsequent training together with the real nodes of the parsing classifier in the current stage. During the training process, the virtual label corresponding to the new virtual node is always 0.

[0111] S602, training the parsing classifier of the previous stage on the mixed data set to obtain the parsing classifier trained in the current stage; during the training process, the optimization function of the parsing classifier parameters is:

[0112]

[0113] Among them, W IL To parse the classifier parameters, Y k is the true label set of the current k-th stage incremental dataset, is the newly added feature set of the current k-th stage incremental data training set, f IL To parse the classifier, represents the predicted label set of the current k-th stage incremental data training set, ||·|| F is the Frobenius norm, λ k-1 is the scaling factor of the initial sampling point set, is the weight of the loss function of the initial sampling point set, is the initial sampling point set, is the initial sampling point set The true label set, is the initial sampling point set The optimization function is the MSE loss function of the mixed dataset, which is used to train the parsing classifier.

[0114] S603, regenerate a set number of sampling points and their corresponding prediction labels; calculate the scaling factor of the current stage based on the set number of sampling points and the amount of data accumulated in the history of the parsing classifier, and the calculation formula is:

[0115]

[0116] Among them, λ k is the scaling factor of the current kth stage, n sam is the set number of sampling points, n kThe amount of data accumulated to parse the classifier history.

[0117] S604, respectively calculating the first-order gradient vector and second-order gradient matrix of the MSE loss function on the mixed data set of the current stage and the regenerated sampling points for the analytical classifier parameters trained in the current stage, and performing gradient alignment and error correction on the regenerated sampling points as the initial sampling point set for the next stage.

[0118] Specifically, the process of gradient alignment and error correction is as follows:

[0119] Calculate the first-order gradient vector g of the MSE loss function on the mixed data set of the current stage for the parameters of the analytical classifier trained in the current stage k and the second-order gradient matrix G k ;

[0120] Calculate the first-order gradient vector of the MSE loss function on the sampling points regenerated in the current stage for the parameters of the analytical classifier trained in the current stage and the second-order gradient matrix

[0121] Use the gradient descent algorithm to optimize the sampling point coordinates so that the second-order gradient matrix Approximately equal to λ k G k Complete gradient alignment; since the calculation of gradients actually requires the coordinates of the sampling points and the corresponding predicted labels, during the optimization process, the sampling points can be directly substituted into the parsing classifier to obtain their corresponding predicted labels;

[0122] The optimized sampling points are input into the analytical classifier trained in the current stage to obtain the predicted labels of the regenerated sampling points, the functional relationship with the corresponding sampling points is cancelled, and the predicted label coordinates of the sampling points are optimized using the gradient descent algorithm so that the first-order gradient vector Approximately equal to λ k g k Complete error correction, i.e., optimize the prediction label;

[0123] Repeat the above steps until all regenerated sampling points are optimized, and the optimized sampling point set is used as the initial sampling point set for the next stage.

[0124] S7, obtaining the incremental data set of the next stage, repeating steps S5-S6 until the set stage is reached or there is no incremental data set of the next stage, and obtaining a learned image classifier.

[0125] Since the actual operation details of the sampling point method depend on the incremental learning part, i.e., f ILThe design of the analytical classifier; Different from the analytical category incremental learning, which directly calculates the required model parameters through matrix operations, the sampling point method needs to take into account the situation of nonlinear models and obtains the results through multiple rounds of gradient descent training, which will inevitably produce errors; When the analytical classifier f IL When it is a linear model, this embodiment conducts a comparative experiment between the sampling point method and the analytical category incremental learning in the same dimension to demonstrate the error of the sampling point method.

[0126] This comparative experiment is trained on the CIFAR-100 dataset, which contains 60,000 32×32 color images of 100 categories, with 500 training images and 100 test images for each category. This comparative experiment uses a 32-layer residual network (ResNet), namely ResNet32, as the background network. This network has cross-layer residual transmission, good resistance to gradient vanishing and network degradation, and has achieved good results in image processing. In the pre-training stage, the stochastic gradient descent algorithm (SGD) is used to train the background network with 160 iterations; the initial learning rate is 0.1, which is divided by 10 at the 80th and 120th iterations; a momentum of 0.9 and a 5×10 -4 The weight decay is 256. The input data is enhanced by random cropping, random horizontal flipping and normalization; the analysis classifier f in the comparative experiment IL The model is set as a linear model, the feature dimension is fixed to 256, and five stages of incremental learning are performed. Two indicators are used to evaluate the performance of the sampling point method:

[0127] Average incremental accuracy (or average accuracy) It is used to evaluate the performance of the sampling point method as a whole. The calculation formula is:

[0128]

[0129] Among them, A k It represents the average accuracy of the parsing classifier tested on the test data set from the 0th to the kth stage after the kth stage of incremental learning. Used to evaluate the overall performance of the algorithm. The higher the value, the better the algorithm performance.

[0130] Gradient sampling error (or gradient error) e k It is the percentage difference between the gradient required at the sampling point and the target gradient, which is divided into the first-order gradient sampling error and the second-order gradient sampling error They are used to measure the accuracy of the predicted labels and sampling points obtained in the kth stage respectively; where:

[0131]

[0132] in, represents the first-order gradient of the MSE loss function on the mixed data set, represents the first-order gradient of the MSE loss function at the sampling point, and ‖‖2 is the Euclidean norm.

[0133]

[0134] in, represents the second-order gradient of the MSE loss function on the mixed data set, Represents the second-order gradient of the MSE loss function at the sampling point; the first-order gradient sampling error and the second-order gradient sampling error The smaller the value, the better the sampling effect.

[0135] This comparative experiment measures the accuracy and error of 8 training processes under different learning rates and iteration numbers. The experimental results are shown in Table 1 below:

[0136] Table 1 Experimental results under different learning rates and iteration numbers

[0137]

[0138]

[0139] The 8 experiments in Table 1 show the performance of three training periods (analytic classifier training, gradient alignment, and error correction) in the incremental learning phase under the sampling point method under three different learning rounds and learning rates. The analytical reference value is the running result of the analytical category incremental learning algorithm when the feature dimension is fixed to 256 (not listed in Table 1). The algorithm performs analytical learning through matrix operations, which is used here to compare the errors generated by the sampling point method. The analytical reference value obtained using the analytical category incremental learning algorithm in this application is 75.57.

[0140] The first thing to be carried out is the training of the parsing classifier. The training effect of the parsing classifier in this stage can be referred to the initial accuracy A0 (%). The above table shows the initial accuracy when the rounds are 200, 225, 250, and 300, and the learning rates are 1 and 1.5 respectively. The experiment with a learning rate of 1 and 250 rounds achieved the lowest accuracy of 75.00; the experiment with a learning rate of 1.5 and 250 rounds achieved the highest accuracy of 75.92. It can be seen that the initial accuracy has random deviations, but the overall gap is not large, and is very close to the parsing reference value of 75.57.

[0141] Gradient alignment needs to be performed after the analytical classifier training is completed. In this stage, the training of the sampling points is completed. The training effect can be referred to the initial second-order gradient error. Table 2 above shows the initial second-order gradient errors for 200 and 250 rounds, and learning rates of 0.01 and 0.015, respectively. The experiments with 200 rounds and a learning rate of 0.01 achieved maximum errors of 0.57 and 0.33, respectively. Although there were random fluctuations in the accuracy of the experiments with 250 rounds and a learning rate of 0.0015, they were all less than 0.3.

[0142] Error correction is performed at the end. This stage completes the training of the labels corresponding to the sampling points and corrects the errors generated in the model training and gradient alignment stages. The accuracy of this stage is greatly affected by the first two stages and can be determined by the initial first-order gradient error. The training effect of this stage was roughly evaluated. The first three experiments showed that when the round was 200 and the learning rate was 0.01, the maximum error was 68.41; when the learning rate was increased from 0.01 to 0.1, the error was reduced from 68.41 to 64.01; when the round was increased from 200 to 250, the error was reduced again to 58.07. When the round was stabilized at 250 and the learning rate was 0.1, the error changed greatly, with the maximum error reaching 60.56 and the minimum being 27.40, depending on the errors generated in the model training stage and the gradient alignment stage. Since the first-order gradient of the loss function of the trained model is usually very close to 0, the first-order gradient error in the above table is much larger than the second-order gradient error.

[0143] Figure 4 , 5 , 6 show the changes in accuracy and gradient error during the incremental learning process of 8 experiments, from the change in accuracy ( Figure 4 ) It can be seen that the initial training accuracy is close to the analytical reference value (75.52), and the difference is less than -1%. After the first stage of incremental learning, the accuracy drops significantly. The difference between this stage and the analytical reference value (67.8) is about -4%. After that, the accuracy is relatively stable in each stage, and the difference with the reference value increases slowly. After the fifth stage of incremental learning, the maximum difference between the accuracy and the analytical reference value (47.25) is about -11.5%, and the minimum difference is about -5.1%. From the gradient error diagram ( Figure 5 , Figure 6 ) It can be clearly seen that the gradient errors generated by the initial sampling points and initial labels obtained in the initial training stage are much larger than those in the subsequent incremental learning stage, among which the first-order gradient error is particularly obvious. The maximum first-order gradient error in the initial training stage can reach 68.41%, while the maximum first-order gradient error in the subsequent incremental learning stage is only 1.58%; the second-order gradient error also has a similar phenomenon. The second-order gradient error in the initial training stage is about 3-4 times the error of the first-stage incremental learning. The gradient error of the initial training is much larger than the gradient error in the subsequent incremental learning stage, which confirms the phenomenon that the accuracy rate decreases more in the first stage of incremental learning.

[0144] To verify the effectiveness of the method in this application on nonlinear models, this application will analyze the classifier f IL The experiment was set as a nonlinear model. In this experiment, the features extracted by the background network were transformed into one dimension and then randomly expanded using the Sigmoid activation function. Then, they were connected to the parsing classifier for classification. Unlike the linear model, the parsing classifier f IL It is a whole composed of nonlinear activation function and linear classifier, and incremental learning is performed on this whole. In order to facilitate comparison with linear performance, the experimental data set, background network and pre-training process in this experiment are consistent with those in the linear experiment. The number of sampling points in this experiment is set to the total size of the second-order gradient matrix (numel) divided by the input dimension of the sampling point (here dim FE ) is rounded up. Due to the large amount of algorithm calculation, this experiment uses dim FE (i.e., sampling point dimension) < 32, and compared with the same dim FE The results are shown in Table 2 below:

[0145] Table 2 Performance of different algorithms in nonlinear model experiments

[0146]

[0147] According to Table 2, in the nonlinear model experiment, the sampling point method showed significantly better performance than the ACIL algorithm, especially in the processing of high-dimensional features. The data in Table 2 show that the accuracy of the sampling point method is higher than that of the ACIL algorithm in each feature dimension, showing its excellent performance in incremental learning of nonlinear models. Although the accuracy of both algorithms decreases with the increase of feature dimensions, the decrease of the sampling point method is significantly smaller, which shows that it has stronger adaptability and robustness when dealing with high-dimensional data. This result shows that the sampling point method optimizes by calculating second-order derivatives and gradients, and does not rely on traditional matrix multiplication and inversion operations, so it has a wider application prospect; it can better handle complex and high-dimensional data in the incremental learning process, and improves the flexibility and generalization ability of the algorithm. In summary, the sampling point method not only provides new research directions in theory, but also demonstrates superior incremental learning capabilities in practice, providing a solid foundation for further research and application.

[0148] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0149] Based on the same idea as the sampling point-based incremental learning method for pseudo-resolution categories of images in the above-mentioned embodiment, the present invention also provides a sampling point-based incremental learning system for pseudo-resolution categories of images, which can be used to execute the sampling point-based incremental learning method for pseudo-resolution categories of images. For ease of explanation, the structural schematic diagram of the embodiment of the sampling point-based incremental learning system for pseudo-resolution categories of images only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0150] like Figure 7 As shown, another embodiment of the present invention provides an image pseudo-analysis category incremental learning system based on sampling points, including a classification network construction module, a background network pre-training module, a virtual node adding module, a parsing classifier pre-training module, a data mixing module, a parsing classifier training module and an incremental learning termination module;

[0151] Among them, the classification network construction module is used to build an image classification network, including a background network and a parsing classifier; the background network extracts image features through a convolutional neural network, and the parsing classifier uses a fully connected network to classify image features and outputs classification results in the form of one-hot encoding; the parsing classifier is constructed based on a linear model or a nonlinear model;

[0152] The background network pre-training module is used to pre-train the background network followed by a softmax classifier on the basic data set using the reverse gradient descent algorithm. After the training, the parameters of the background network are frozen and the one-dimensional features of the basic data set are obtained.

[0153] The virtual node adding module is used to remove the softmax classifier, connect the parsing classifier after the background network with frozen parameters, and add a virtual node to the parsing classifier to participate in subsequent operations; the corresponding classification result adds a dimension, which is the virtual label of the corresponding virtual node that is always 0;

[0154] The parsing classifier pre-training module is used to perform random dimensional expansion on the one-dimensional features of the basic data set. The input parsing classifier uses the reverse gradient descent algorithm to perform gradient realignment. After completion, the scaling coefficient is calculated and the initial sampling point set is obtained. The number of sampling points in the initial sampling point set is determined by the parsing classifier and kept constant.

[0155] The data mixing module is used to obtain the incremental data set of the current stage, input the background network of frozen parameters to obtain the newly added feature set, and mix it with the initial sampling point set to obtain the mixed feature set;

[0156] The parsing classifier training module is used to perform incremental learning on the parsing classifier of the previous stage on the mixed feature set by using the MSE loss function through the reverse gradient descent algorithm, and update the sampling points as the initial sampling point set of the next stage;

[0157] The incremental learning termination module is used to obtain the incremental data set for the next stage, and repeatedly execute the data mixing module and the parsing classifier training module until the set stage is reached or there is no incremental data set for the next stage, and a learned image classifier is obtained.

[0158] It should be noted that the sampling point-based image pseudo-resolution category incremental learning system of the present invention corresponds one-to-one to the sampling point-based image pseudo-resolution category incremental learning method of the present invention. The technical features and beneficial effects described in the embodiment of the above-mentioned sampling point-based image pseudo-resolution category incremental learning method are applicable to the embodiment of the sampling point-based image pseudo-resolution category incremental learning system. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. This is hereby declared.

[0159] In addition, in the implementation of the sampling point-based incremental learning system for pseudo-resolution categories of images in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the sampling point-based incremental learning system for pseudo-resolution categories of images can be divided into different program modules to complete all or part of the functions described above.

[0160] like Figure 8 As shown, in one embodiment, an electronic device for implementing a sampling point-based incremental learning method for pseudo-resolution categories of images is provided. The electronic device may include a first processor, a first memory, and a bus, and may also include a computer program stored in the first memory and executable on the first processor, such as a sampling point-based incremental learning program for pseudo-resolution categories of images.

[0161] Among them, the first memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the first memory can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the first memory can also include both an internal storage unit of the electronic device and an external storage device. The first memory can not only be used to store application software and various types of data installed in the electronic device, such as the code of the incremental learning program of the pseudo-analysis category of the image based on the sampling point, but also can be used to temporarily store data that has been output or is to be output.

[0162] In some embodiments, the first processor may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips, etc. The first processor is the control core (ControlUnit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the first memory (such as an incremental learning program for pseudo-analysis categories of images based on sampling points, etc.), and calls data stored in the first memory to perform various functions of the electronic device and process data.

[0163] Figure 8 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0164] The sampling point-based image pseudo-analysis category incremental learning program stored in the first memory of the electronic device is a combination of multiple instructions, and when executed in the first processor, can achieve:

[0165] S1. Construct an image classification network, including a background network and a parsing classifier; wherein the background network extracts image features through a convolutional neural network, and the parsing classifier uses a fully connected network to classify image features and outputs classification results in a one-hot encoding form; the parsing classifier is constructed based on a linear model or a nonlinear model;

[0166] S2. Use the reverse gradient descent algorithm to pre-train the background network followed by the softmax classifier on the basic data set. After the training, freeze the parameters of the background network and obtain the one-dimensional features of the basic data set.

[0167] S3, remove the softmax classifier, connect the parsing classifier after the background network with frozen parameters, and add a virtual node in the parsing classifier to participate in subsequent operations; the corresponding classification result adds a dimension, which is the virtual label of the corresponding virtual node that is always 0;

[0168] S4, performing random dimensional expansion on the one-dimensional features of the basic data set, inputting the analytic classifier to perform gradient realignment using the reverse gradient descent algorithm, and calculating the scaling coefficient after completion to obtain an initial sampling point set; wherein the number of sampling points in the initial sampling point set is determined by the analytic classifier and kept constant;

[0169] S5, obtaining the incremental data set of the current stage, inputting the background network of the frozen parameters to obtain the newly added feature set, and mixing it with the initial sampling point set to obtain a mixed data set;

[0170] S6, using the MSE loss function to perform incremental learning on the parsing classifier of the previous stage on the mixed data set through the reverse gradient descent algorithm, and updating the sampling points as the initial sampling point set of the next stage;

[0171] S7, obtaining the incremental data set of the next stage, repeating steps S5-S6 until the set stage is reached or there is no incremental data set of the next stage, and obtaining a learned image classifier.

[0172] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0173] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An incremental learning method for pseudo-analytic categories of images based on sampling points, characterized in that: The steps include: S1. Construct an image classification network, including a background network and a parsing classifier; the background network extracts image features through a convolutional neural network, and the parsing classifier uses a fully connected network to classify image features and outputs classification results in a one-hot encoding form; the parsing classifier is constructed based on a linear model or a nonlinear model; S2. Use the reverse gradient descent algorithm to pre-train the background network followed by the softmax classifier on the basic data set. After the training, freeze the parameters of the background network and obtain the one-dimensional features of the basic data set. S3, remove the softmax classifier, connect the parsing classifier after the background network with frozen parameters, and add a virtual node in the parsing classifier to participate in subsequent operations; the corresponding classification result adds a dimension, which is the virtual label of the corresponding virtual node that is always 0; S4, performing random dimensional expansion on the one-dimensional features of the basic data set, inputting the features into the analytical classifier and realigning them using the reverse gradient descent algorithm, and calculating the scaling coefficient after completion to obtain an initial sampling point set; the number of sampling points in the initial sampling point set is determined by the analytical classifier and kept constant; S5, obtaining the incremental data set of the current stage, inputting the background network of the frozen parameters to obtain the newly added feature set, and mixing it with the initial sampling point set to obtain a mixed data set; S6, using the MSE loss function to perform incremental learning on the parsing classifier of the previous stage on the mixed data set through the reverse gradient descent algorithm, and updating the sampling points as the initial sampling point set of the next stage; The incremental learning of the parsing classifier in the previous stage on the mixed data set is specifically as follows: Based on the number of newly added labels in the incremental data set at the current stage, the constructed virtual nodes and their corresponding parsing classifier parameters at the current stage are copied, and the last copied virtual node is used as a new virtual node, and the remaining copied virtual nodes are used as real nodes and the real nodes of the parsing classifier at the current stage to participate in subsequent incremental learning; the number of copies is equal to the number of newly added labels; the virtual label corresponding to the new virtual node is always 0; Train the parsing classifier of the previous stage on the mixed data set to obtain the parsing classifier trained in the current stage; Regenerate a set number of sample points and their corresponding predicted labels; Calculate the scaling factor for the current stage based on the set number of sampling points and the amount of data accumulated in the history of the parsing classifier; Calculate the first-order gradient vector and second-order gradient matrix of the MSE loss function on the mixed data set of the current stage and the regenerated sampling points for the parameters of the analytical classifier trained in the current stage, and perform gradient alignment and error correction on the regenerated sampling points as the initial sampling point set for the next stage; S7, obtaining the incremental data set of the next stage, repeating steps S5-S6 until the set stage is reached or there is no incremental data set of the next stage, and obtaining a learned image classifier.

2. The image pseudo-analysis category incremental learning method based on sampling points as claimed in claim 1, characterized in that: In step S2, the process of pre-training the background network followed by the softmax classifier is as follows: The basic data set is input into the background network for feature extraction to obtain the basic data feature set; Flatten the basic data feature set in one dimension to obtain the one-dimensional features of the basic data set; The subsequent softmax classifier classifies the one-dimensional features of the basic data set and outputs the classification results in the form of one-hot encoding, expressed as: Z = f softmax (f flat (f CNN (X0,W CNN ),W FCN )); where f softmax is the softmax classifier, f flat is a one-dimensional flattening operation, f CNN is the background network, X0 is the basic data set, W CNN is the background network parameter, W FCN is the parameter of the linear output layer in the one-dimensional flattening operation, which adjusts the dimension of the one-dimensional feature to the same dimension as the true label of the basic dataset; The loss function is calculated based on the classification results and the true labels of the basic data set, and the network parameters are updated through the reverse gradient descent algorithm until the loss function converges or the maximum number of iterations is reached, freezing the parameters of the background network.

3. The image pseudo-analysis category incremental learning method based on sampling points as claimed in claim 1, characterized in that: In step S4, the input parsing classifier is realigned using a reverse gradient descent algorithm, the steps are: Use the activation function to randomly expand the one-dimensional features of the basic data set to obtain the expanded feature set Among them, f act is the activation function, f flat is a one-dimensional flattening operation, f CNN is the background network, X0 is the basic data set, W CNN is the background network parameter, W FCN is the parameter of the linear output layer in the one-dimensional flattening operation, which adjusts the dimension of the one-dimensional feature to the same dimension as the true label of the basic dataset, W FE Expand the matrix for randomly generated parameters; The expanded feature set is input into the parsing classifier and realigned using the MSE loss function through the reverse gradient descent algorithm; during the realignment process, the optimization function of the parsing classifier parameters is: Among them, W IL To analyze the parameters of the classifier, Y0 is the true label set of the basic data set, f IL To parse the classifier, represents the predicted label set of the basic dataset, ||·|| F is the Frobenius norm; After completion, a set number of sampling points and their corresponding prediction labels are randomly generated, the sampling points and their corresponding prediction labels are optimized based on the expanded feature set and prediction label set, the scaling coefficient is calculated and the initial sampling point set is obtained.

4. The image pseudo-analysis category incremental learning method based on sampling points as claimed in claim 3, characterized in that: The steps for obtaining the initial sampling point set are: A set number of sampling points and their corresponding predicted labels are randomly generated; the coordinates of the sampling points and their corresponding predicted labels are randomly generated; Based on the set number of sampling points and the number of basic data sets, the scaling factor λ0 is calculated as follows: Among them, n sam is the set number of sampling points, n0 is the number of basic data sets; Calculate the first-order gradient vector g0 and second-order gradient matrix G0 of the MSE loss function on the basic data set for the parameters of the trained analytic classifier, as well as the first-order gradient vector of the MSE loss function on the sampling point for the parameters of the trained analytic classifier. and the second-order gradient matrix Use the gradient descent algorithm to optimize the sampling point coordinates and their corresponding prediction labels respectively, so that the second-order gradient matrix is approximately equal to λ0G0 and the first-order gradient vector Approximately equal to λ0g0, the initial sampling point set is obtained.

5. The image pseudo-analysis category incremental learning method based on sampling points as claimed in claim 1, characterized in that: During the training process, the optimization function of the analytical classifier parameters is: Among them, W IL To parse the classifier parameters, Y k is the true label set of the current k-th stage incremental dataset, is the newly added feature set of the current k-th stage incremental data set, f IL To parse the classifier, represents the predicted label set of the current k-th stage incremental dataset, ||·|| F is the Frobenius norm, λ k-1 is the scaling factor of the initial sampling point set, is the weight of the loss function of the initial sampling point set, is the initial sampling point set, is the initial sampling point set The true label set, is the initial sampling point set The predicted label set of The calculation formula of the scaling factor is: Among them, λ k is the scaling factor of the current kth stage, n sam is the set number of sampling points, n k The amount of data accumulated to parse the classifier history.

6. The image pseudo-analysis category incremental learning method based on sampling points as claimed in claim 1, characterized in that: The regenerated sampling points are subjected to gradient alignment and error correction as the initial sampling point set for the next stage, specifically: Calculate the first-order gradient vector g of the MSE loss function on the mixed data set of the current stage for the parameters of the analytical classifier trained in the current stage k and the second-order gradient matrix G k ; Calculate the first-order gradient vector of the MSE loss function on the sampling points regenerated in the current stage for the parameters of the analytical classifier trained in the current stage and the second-order gradient matrix Use the gradient descent algorithm to optimize the sampling point coordinates so that the second-order gradient matrix Approximately equal to λ k G k Complete gradient alignment; The optimized sampling points are input into the analytical classifier trained in the current stage to obtain the predicted labels of the regenerated sampling points, and the gradient descent algorithm is used to optimize the predicted label coordinates of the sampling points so that the first-order gradient vector Approximately equal to λ k g k Complete error correction; Repeat the above steps until all regenerated sampling points are optimized, and the optimized sampling point set is used as the initial sampling point set for the next stage.

7. The image pseudo-analysis category incremental learning system based on sampling points is characterized by: It includes a classification network construction module, a background network pre-training module, a virtual node addition module, a parsing classifier pre-training module, a data mixing module, a parsing classifier training module and an incremental learning termination module; The classification network construction module is used to construct an image classification network, including a background network and a parsing classifier; the background network extracts image features through a convolutional neural network, and the parsing classifier uses a fully connected network to classify image features and outputs classification results in a one-hot encoding form; the parsing classifier is constructed based on a linear model or a nonlinear model; The background network pre-training module is used to pre-train the background network followed by a softmax classifier on the basic data set using a reverse gradient descent algorithm, and after completion, the parameters of the background network are frozen and one-dimensional features of the basic data set are obtained; The virtual node adding module is used to remove the softmax classifier, connect the parsing classifier after the background network with frozen parameters, and add a virtual node in the parsing classifier to participate in subsequent operations; the corresponding classification result adds a dimension, which is the virtual label of the corresponding virtual node that is always 0; The analytical classifier pre-training module is used to perform random dimensional expansion on the one-dimensional features of the basic data set, input the analytical classifier to perform realignment using the reverse gradient descent algorithm, and after completion, calculate the scaling coefficient and obtain an initial sampling point set; the number of sampling points in the initial sampling point set is determined by the analytical classifier and kept constant; The data mixing module is used to obtain the incremental data set of the current stage, input the background network of the frozen parameters to obtain the newly added feature set, and mix it with the initial sampling point set to obtain a mixed feature set; The parsing classifier training module is used to perform incremental learning on the parsing classifier of the previous stage on the mixed feature set by using the MSE loss function through the reverse gradient descent algorithm, and update the sampling points as the initial sampling point set of the next stage; The incremental learning of the parsing classifier in the previous stage on the mixed data set is specifically as follows: Based on the number of newly added labels in the incremental data set at the current stage, the constructed virtual nodes and their corresponding parsing classifier parameters at the current stage are copied, and the last copied virtual node is used as a new virtual node, and the remaining copied virtual nodes are used as real nodes and the real nodes of the parsing classifier at the current stage to participate in subsequent incremental learning; the number of copies is equal to the number of newly added labels; the virtual label corresponding to the new virtual node is always 0; Train the parsing classifier of the previous stage on the mixed data set to obtain the parsing classifier trained in the current stage; Regenerate a set number of sample points and their corresponding predicted labels; Calculate the scaling factor for the current stage based on the set number of sampling points and the amount of data accumulated in the history of the parsing classifier; Calculate the first-order gradient vector and second-order gradient matrix of the MSE loss function on the mixed data set of the current stage and the regenerated sampling points for the parameters of the analytical classifier trained in the current stage, and perform gradient alignment and error correction on the regenerated sampling points as the initial sampling point set for the next stage; The incremental learning termination module is used to obtain the incremental data set of the next stage, repeatedly execute the data mixing module and the analytical classifier training module until the set stage is reached or there is no incremental data set of the next stage, and obtain a learned image classifier.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory in communication with the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the sampling point-based image pseudo-parse category incremental learning method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the sampling point-based image pseudo-analysis category incremental learning method described in any one of claims 1 to 6 is implemented.

Images