Generalized learning mechanism-based regularization continuous learning method

Abstract

The invention discloses a regularization continuous learning method based on a generalization learning mechanism, and relates to the technical field of machine learning, and the method comprises the following steps: S1, carrying out the pattern conversion of an image set Dt, and generating a shape-texture conflict image # imgabs0 #: S2, extracting the image features through a neural network model theta t, and calculating the KL divergence loss LSTCR through a consistency regularization mechanism; s3, calculating the total loss Lt and optimizing the neural network model theta t; the model can learn shape and texture features at the same time in the training process, the generalization ability of the model is remarkably improved, it is ensured that the model keeps knowledge about old tasks while learning new tasks, the disastrous forgetting problem is effectively relieved, meanwhile, dependence on external storage is avoided, extra large-scale storage resources are not needed in the training process, and the training efficiency is improved. And the calculation and storage cost of the whole system is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a regularization continuous learning method based on a generalization learning mechanism. Background Art

[0002] Although the prior art has made certain progress in the fields of generalization learning and continuous learning, there are still problems such as insufficient coverage of data diversity, insufficient knowledge retention, and high storage and computing costs. These drawbacks limit the performance and efficiency of the model in practical applications;

[0003] Existing data augmentation methods mainly rely on basic geometric transformations, such as rotation, scaling, cropping, etc., and cannot comprehensively cover complex real-world data distributions. This results in limited generalization ability of the model when facing diverse new tasks, and it is difficult to effectively handle unseen samples and different data distributions;

[0004] Existing regularization methods and replay methods have limitations in preventing catastrophic forgetting, and cannot effectively balance new task learning and old task knowledge retention, restricting the model's learning ability for new tasks. The replay method relies on storing a large number of old task samples and is difficult to remain efficient when the number of tasks increases;

[0005] Existing replay methods need to store a large number of old task data samples. As the number of tasks increases, the storage requirements and computing costs increase linearly. This not only occupies a large amount of storage space but also increases the complexity of data management and processing, restricting its application in resource-constrained environments.

[0006] Therefore, a regularization continuous learning method based on a generalization learning mechanism is provided to solve the above problems. Summary of the Invention

[0007] The object of the present invention is to provide a regularization continuous learning method based on a generalization learning mechanism, which simultaneously learns shape features and texture features, keeps the important weights of old tasks unchanged when learning new tasks, further enhances the knowledge retention ability, improves the model generalization ability and continuous learning while reducing the computing and storage costs.

[0008] To achieve the above object, the present invention provides a regularization continuous learning method based on a generalization learning mechanism, including the following steps:

[0009] S1: Perform style conversion on the image set D t to generate shape-texture conflict images

[0010] S2: Extract image features through the neural network model Θ t and calculate the KL divergence loss L through the consistency regularization mechanism STCR, and minimize the KL divergence loss L STCR ;

[0011] S3: Calculate the overall loss L t and optimize the neural network model Θ t .

[0012] Preferably, step S1 specifically includes the following steps:

[0013] S11: Randomly sample two mini-batch images from the image set D t . Both of the two mini-batch images include k images. The two mini-batch images are the content image x and the style image The content image x is set as x = {x 1,t ,..., x k,t}, and the style image is set as

[0014] S12: Combine the content image x and the style image through the real-time style transfer algorithm AdaIN in the style transfer module to generate a shape-texture conflict image The shape-texture conflict image is denoted as

[0015] Preferably, in step S12, the conversion formula of the real-time style transfer algorithm AdaIN is:

[0016]

[0017] where f(·) is the style conversion operation.

[0018] Preferably, step S2 specifically includes the following steps:

[0019] S21: Input the content image x and the shape-texture conflict image into the neural network model Θ t to respectively extract the logits z and the logits The logits z is set as z = {z 1,t ,..., z k,t}, and the logits is set as

[0020] S22: Calculate the KL divergence loss L between the logits z and the logits STCR .

[0021] Preferably, in step S21, the encoded content of the logit z is the texture information of the image, and the logit encoded content is the shape information of the image.

[0022] Preferably, in step S22, the KL divergence loss L STCR is calculated as follows:

[0023] L STCR = D KL (p|q + D KL (q||p)

[0024] p = σ(z / τ)

[0025]

[0026] where τ is the temperature hyperparameter and σ(·) is the Softmax function.

[0027] Preferably, step S3 specifically includes the following steps:

[0028] S31: Calculate the overall loss L based on the cross-entropy loss of the memory buffer P CE of the cross-entropy loss L STCR and the KL divergence loss L t ;

[0029] S32: Minimize the overall loss L t through the optimization module;

[0030] S33: Update the neural network model Θ t .

[0031] Preferably, in step S31, the formula for calculating the overall loss L t is:

[0032]

[0033] where β and γ both represent weight coefficients.

[0034] Therefore, the present invention adopts the above regularization continuous learning method based on the generalization learning mechanism, and has the following beneficial effects:

[0035] (1) By introducing the shape-texture consistency regularization method and using the style transfer technology to generate shape-texture conflict images, the present invention not only contains diverse texture information but also integrates shape features, enabling the model to simultaneously learn shape and texture features during the training process, significantly improving the generalization ability of the model and being able to more effectively handle unseen samples and different data distributions;

[0036] (2) The present invention ensures that the model retains knowledge of old tasks while learning new tasks through a consistency regularization mechanism. By minimizing L STCR , it ensures that the model maintains memory of the knowledge of old tasks when learning new tasks, thus effectively alleviating the catastrophic forgetting problem;

[0037] (3) The STCR method of the present invention can be seamlessly integrated with the replay method. Using the samples in the memory buffer P as style templates further enhances the knowledge retention ability and avoids dependence on external storage;

[0038] (4) The present invention adopts the efficient real-time style transfer algorithm AdaIN for online calculation design, which eliminates the need for additional large-scale storage resources during training, further reducing the overall system's computational and storage costs, enhancing the practical application feasibility of the method, and being applicable to resource-constrained continuous learning environments.

[0039] The method solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0040] Figure 1 is a flowchart of a regularization continuous learning method based on a generalization learning mechanism of the present invention;

[0041] Figure 2 is the generation process of shape-texture conflict images in an embodiment of the present invention. Detailed Embodiments

[0042] The method solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0043] Unless otherwise defined, the method terms or scientific terms used in the present invention shall have the ordinary meanings understood by those of ordinary skill in the field to which the present invention pertains.

[0044] In the present invention, words such as "including" or "comprising" mean that the elements before this word cover the elements listed after this word, and do not exclude the possibility of also covering other elements. The orientation or positional relationship indicated by terms such as "inside", "outside", "above", "below", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation on the present invention. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In the present invention, unless otherwise clearly specified and defined, terms such as "attachment" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be directly connected, or indirectly connected through an intermediate medium. It can be the communication inside two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0045] Embodiment

[0046] As Figure 1 and Figure 2 shown, the present invention provides a regularization continuous learning method based on a generalization learning mechanism, including the following steps:

[0047] S1: Perform style conversion on the image set D t to generate shape-texture conflict images

[0048] Step S1 specifically includes the following steps:

[0049] S11: Randomly sample two mini-batch images from the image set D t . Both of the two mini-batch images include k images. The two mini-batch images are a content image x and a style image The content image x is set as x = {x 1,t ,..., x k,t}, and the style image is set as

[0050] S12: Combine the content image x and the style image through the real-time style conversion algorithm AdaIN in the style conversion module to generate shape-texture conflict images The shape-texture conflict image is expressed as

[0051] In step S12, the conversion formula of the real-time style conversion algorithm AdaIN is as follows:

[0052]

[0053] where f(·) is the style conversion operation, and the pre-trained real-time style conversion algorithm AdaIN is adopted to ensure the generated shape-texture conflict image can effectively fuse shape and texture features.

[0054] S2: Extract image features through the neural network model Θ t and calculate the KL divergence loss L through the consistency regularization mechanism STCR to ensure that the neural network model Θ t can simultaneously learn shape features and texture features, and minimize the KL divergence loss L STCR to prompt the neural network model Θ t to maintain the memory of the knowledge of the old task when learning a new task, and enhance the generalization ability of the neural network model Θ t ;

[0055] Step S2 specifically includes the following steps:

[0056] S21: Input the content image x and the shape-texture conflict image into the neural network model Θ t to extract the logits z and the logits The logits z are set as z = {z 1,t ,..., z k,t}, and the logits are set as

[0057] In step S21, the encoded content of the logits z is the texture information of the image, and the encoded content of the logits is the shape information of the image;

[0058] S22: Calculate the KL divergence loss L between the logits z and the logits STCR through the consistency regularization mechanism;

[0059] In step S22, the calculation formula of the KL divergence loss L STCR is as follows:

[0060] L STCR = D KL (p||q + D KL (q||p)

[0061] p = σ(z / τ)

[0062]

[0063] Among them, τ is the temperature hyperparameter, and σ(·) is the Softmax function.

[0064] S3: Calculate the overall loss L t and optimize the neural network model Θ t , and at the same time optimize the neural network model Θ t for its classification performance and consistency regularization effect;

[0065] Step S3 specifically includes the following steps:

[0066] S31: According to the cross-entropy loss of the current task the cross-entropy loss L CE P of the memory buffer P and the KL divergence loss L STCR calculate the overall loss L t ;

[0067] In step S31, the calculation formula of the overall loss L t is:

[0068]

[0069] where both β and γ represent weight coefficients used to balance the influence of different loss terms;

[0070] S32: Minimize the overall loss L t through the optimization module;

[0071] S33: Update the neural network model Θ t , to ensure that the neural network model Θ t has good generalization ability and memory retention ability when learning new tasks.

[0072] Therefore, the present invention adopts the above-mentioned regularization continuous learning method based on a generalization learning mechanism, simultaneously learns shape features and texture features, retains the memory of old task knowledge when learning new tasks, effectively solves the problem of catastrophic forgetting, reduces the computational and storage costs, and effectively solves the problems of insufficient coverage of data diversity, insufficient knowledge retention, and high computational costs in the prior art.

[0073] Finally, it should be noted that: the above embodiments are only used to illustrate the method solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that: they can still modify or equivalently replace the method solution of the present invention, and these modifications or equivalent replacements cannot make the modified method solution deviate from the spirit and scope of the method solution of the present invention.

Claims

1. A regularized continuous learning method based on a generalized learning mechanism, characterized in that: The following steps are involved: S1: for image set D t Perform style transfer to generate shape-texture conflict images S2: Through the neural network model Θ t Extract image features and calculate KL divergence loss L through consistency regularization mechanism STCR , and the KL divergence loss L STCR Minimize the processing; S3: Calculate the overall loss L t And optimize the neural network model Θ t .

2. A regularized continuous learning method based on a generalized learning mechanism according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11: From image set D t Randomly sample two mini-batch images from the image, each of which includes k images. The two mini-batch images are content image x and style image x. The content image x is set to x={x 1,t ,...,x k,t }, style image Set to S12: The content image x and the style image x are transformed by the real-time style transformation algorithm AdaIN in the style transformation module. Combine to generate shape-texture conflict image Shape-Texture Conflict Image Expressed as 3. A regularized continuous learning method based on a generalized learning mechanism according to claim 2, characterized in that: In step S12, the conversion formula of the real-time style conversion algorithm AdaIN is: where f(·) is the style transfer operation.

4. The regularized continuous learning method based on a generalized learning mechanism according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21: Content image x and shape-texture conflict image Input to the neural network model Θ t In the above example, we extract the logarithmic probability z and logarithmic probability The logarithmic probability z is set to z = {z 1,t ,...,z k,t }, log odds Set to S22: Computing log-odds z and log-odds via consistency regularization The KL divergence loss L between STCR .

5. A regularized continuous learning method based on a generalized learning mechanism according to claim 4, characterized in that: In step S21, the content encoded by the logarithmic probability z is the texture information of the image. The encoded content is the shape information of the image.

6. A regularized continuous learning method based on a generalized learning mechanism according to claim 4, characterized in that: In step S22, the KL divergence loss L STCR The calculation formula is: L STCR =D KL (p||q)+D KL (q||p) p=σ(z / τ) Where τ is the temperature hyperparameter and σ(·) is the Softmax function.

7. The regularized continuous learning method based on a generalized learning mechanism according to claim 1, characterized in that: Step S3 specifically includes the following steps: S31: Cross entropy loss according to the current task The cross entropy loss L of the memory buffer P CE (P) and KL divergence loss L STCR Calculate the overall loss L t ; S32: The overall loss L is optimized by the module t Minimize the processing; S33: Update the neural network model Θ t .

8. A regularized continuous learning method based on a generalized learning mechanism according to claim 7, characterized in that: In step S31, the overall loss L t The calculation formula is: Where β and γ are weight coefficients.

Images