Comparative learning image rain removal model training method, rain removal method and storage medium

Abstract

The invention discloses a contrast learning image rain removal model training method, a rain removal method and a storage medium. The model training method comprises the steps of obtaining an original rain image and inputting the original rain image into an initialized rain removal model to obtain initial data; the background layer and the rain layer are alternately fixed based on the initial data to iteratively optimize the rain removal model, high-dimensional features of the background layer and the rain layer after iteration optimization are extracted based on the feature extraction network, a comparison loss value is calculated, and the high-dimensional features comprise background layer features and rain layer features; calculating a total loss value based on the high-dimensional features, the original rain image and the comparison loss value, and updating model parameters based on back propagation of the model loss value; and stopping iteration when the number of iterations reaches a preset value or the total loss value converges, and outputting a final rain-removed image and a final rain layer. According to the method, the rain layer and the image layer are alternately iterated and solved, the loss constraints are compared, the rain layer and the image layer can be better distinguished, and finally separation of the rain layer and the image layer is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically to a comparative learning image rain removal model training method, a rain removal method and a storage medium. Background Art

[0002] With the rapid development and widespread application of artificial intelligence and computer vision, high-quality images have become the foundation of vision systems for intelligent surveillance, autonomous driving, and other applications. However, in practical applications, weather conditions such as rain, snow, and fog not only blur images and reduce contrast, but also produce false texture features, seriously affecting subsequent image analysis and decision-making.

[0003] With the rapid development of deep learning technology, various innovative neural network architectures have emerged. The current mainstream deraining methods mainly adopt end-to-end supervised learning strategies, training the network by constructing a pixel-level mapping relationship between rainy images and rain-free images.

[0004] However, these methods usually directly impose constraints on the background layer after rain removal to make it as close as possible to the real rain-free image. Single-dimensional constraints are often difficult to fully capture the complex interactive relationship between rain streaks and the background. Summary of the Invention

[0005] The purpose of the present invention is to provide a contrastive learning image rain removal model training method, a rain removal method and a storage medium. By alternately iteratively solving the rain layer and the image layer and performing contrast loss constraints, the rain layer and the image layer can be better distinguished, and ultimately the rain layer and the image layer can be separated.

[0006] A first aspect of the present invention provides a contrastive learning image deraining model training method, comprising the following steps:

[0007] Obtaining an original rainy image and inputting it into an initialized rain removal model to obtain initial data, wherein the initial data includes an initial background layer and an initial rain layer;

[0008] The background layer and the rain layer are alternately fixed based on the initial data to iteratively optimize the rain removal model, wherein the rain removal model is composed of the background layer and the rain layer,

[0009] Extracting high-dimensional features of the iteratively optimized background layer and rain layer based on a feature extraction network and calculating a contrast loss value, wherein the high-dimensional features include background layer features and rain layer features;

[0010] Calculating a total loss value based on the high-dimensional features, the original rainy image, and the contrast loss value, and updating model parameters based on the model loss value through backpropagation;

[0011] When the number of iterations reaches the preset value or the total loss value converges, the iteration stops and the final derained image and rain layer are output.

[0012] In this solution, the objective function of the rain removal model is as follows:

[0013]

[0014] in, is the reconstruction error term, which is used to decompose the input original rainy image into the background layer I and the rain layer R. and It is the regularization prior applied to the background layer and rain layer of the image, J is the original rainy image, I is the background layer to be optimized, and R is the rain layer to be optimized.

[0015] In this solution, the rain layer is fixed and the background layer is iteratively optimized, specifically including:

[0016] Subproblem of calculating the background layer:

[0017]

[0018] in, I k +1 is the updated background layer, I is the background layer to be optimized, R k is the current rain layer, J is the original rainy image, is the regularization prior applied to the image background layer;

[0019] Calculate the background layer residual gradient:

[0020] r I =I k +ρ(JI k -R k );

[0021]

[0022] Among them, r I is the background layer residual gradient, ρ is the control step size, I k is the current background layer, R k is the current rain layer, J is the original rainy image, I k+1 is the updated background layer, I is the background layer to be optimized, is the regularization prior applied to the image background layer;

[0023] The background layer after iterative update is:

[0024]

[0025] Among them, I k+1 For the updated background layer, is the background layer extraction function, r I is the residual gradient of the background layer.

[0026] In this solution, the background layer is fixed and the rain layer is iteratively optimized, specifically including:

[0027] Subproblem of computing the rain layer:

[0028]

[0029] Among them, R k+1 For the updated rain layer, I k+1 is the updated background layer, R is the rain layer to be optimized, and J is the original rainy image. is the regularization prior applied to the rain layer of the image;

[0030] Calculate the residual gradient of the rain layer:

[0031] r R =R k +ρ(JI k+1 -R k );

[0032]

[0033] Among them, r R is the residual gradient of the rain layer, R k is the current rain layer, ρ is the control step size, R k+1 is the updated rain layer, R k is the current rain layer, J is the original rainy image, I k+1 is the updated background layer, R is the rain layer to be optimized, is the regularization prior applied to the rain layer of the image;

[0034] The rain layer after iterative update is:

[0035]

[0036] Among them, R k+1 For the updated rain layer, is the rain layer extraction function, r R is the residual gradient of the rain layer.

[0037] In this solution, the contrast loss value is calculated by extracting high-dimensional features of the iteratively optimized background layer and rain layer based on the feature extraction network, specifically including:

[0038] Use the feature extraction network f(·) to extract the background layer features f I and rain layer characteristics f R ,in,

[0039] f I =f(I k+1 ), f R =f(Rk+1 );

[0040] Maximize the background layer feature f I and rain layer characteristics f R The difference between and is used to calculate the contrast loss value:

[0041]

[0042] Among them, L contrast is the contrast loss value, f I is the background layer feature, f R is the rain layer feature, s(·) is the similarity function, τ is the temperature coefficient, and N is the number of negative samples.

[0043] In this solution, the total loss value is calculated based on the high-dimensional features, the original rainy image, and the contrast loss value, specifically including:

[0044] Calculate the model loss value:

[0045]

[0046] Among them, L model is the model loss value, is the reconstruction error term, J is the original rainy image, I is the background layer to be optimized, R is the rain layer to be optimized, is the regularization loss of the background layer, ‖R‖1 is the sparse loss of the rain layer;

[0047] Calculate the total loss value:

[0048] L total =L model +λL contrast ;

[0049] Among them, L total is the total loss value, L model is the model loss value, L contrast is the contrast loss value, and λ is the loss weight.

[0050] A second aspect of the present invention further provides a contrastive learning image deraining model training system, comprising a memory and a processor, wherein the memory comprises a contrastive learning image deraining model training method program, and when the contrastive learning image deraining model training method program is executed by the processor, the following steps are implemented:

[0051] Obtaining an original rainy image and inputting it into an initialized rain removal model to obtain initial data, wherein the initial data includes an initial background layer and an initial rain layer;

[0052] The background layer and the rain layer are alternately fixed based on the initial data to iteratively optimize the rain removal model, wherein the rain removal model is composed of the background layer and the rain layer,

[0053] Extracting high-dimensional features of the iteratively optimized background layer and rain layer based on a feature extraction network and calculating a contrast loss value, wherein the high-dimensional features include background layer features and rain layer features;

[0054] Calculating a total loss value based on the high-dimensional features, the original rainy image, and the contrast loss value, and updating model parameters based on the model loss value through backpropagation;

[0055] When the number of iterations reaches the preset value or the total loss value converges, the iteration stops and the final derained image and rain layer are output.

[0056] In this solution, the objective function of the rain removal model is as follows:

[0057]

[0058] in, is the reconstruction error term, which is used to decompose the input original rainy image into the background layer I and the rain layer R. and It is the regularization prior applied to the background layer and rain layer of the image, J is the original rainy image, I is the background layer to be optimized, and R is the rain layer to be optimized.

[0059] In this solution, the rain layer is fixed and the background layer is iteratively optimized, specifically including:

[0060] Subproblem of calculating the background layer:

[0061]

[0062] Among them, I k+1 is the updated background layer, I is the background layer to be optimized, R k is the current rain layer, J is the original rainy image, is the regularization prior applied to the image background layer;

[0063] Calculate the background layer residual gradient:

[0064] r I =I k +ρ(JI k -R k );

[0065]

[0066] Among them, r I is the background layer residual gradient, ρ is the control step size, I k is the current background layer, R k is the current rain layer, J is the original rainy image, I k+1is the updated background layer, I is the background layer to be optimized, is the regularization prior applied to the image background layer;

[0067] The background layer after iterative update is:

[0068]

[0069] Among them, I k+1 For the updated background layer, is the background layer extraction function, r I is the residual gradient of the background layer.

[0070] In this solution, the background layer is fixed and the rain layer is iteratively optimized, specifically including:

[0071] Subproblem of computing the rain layer:

[0072]

[0073] Among them, R k+1 For the updated rain layer, I k+1 is the updated background layer, R is the rain layer to be optimized, and J is the original rainy image. is the regularization prior applied to the rain layer of the image;

[0074] Calculate the residual gradient of the rain layer:

[0075] r R =R k +ρ(JI k+1 -R k );

[0076]

[0077] Among them, r R is the residual gradient of the rain layer, R k is the current rain layer, ρ is the control step size, R k+1 is the updated rain layer, R k is the current rain layer, J is the original rainy image, I k+1 is the updated background layer, R is the rain layer to be optimized, is the regularization prior applied to the rain layer of the image;

[0078] The rain layer after iterative update is:

[0079]

[0080] Among them, R k+1 For the updated rain layer, is the rain layer extraction function, r R is the residual gradient of the rain layer.

[0081] In this solution, the contrast loss value is calculated by extracting high-dimensional features of the iteratively optimized background layer and rain layer based on the feature extraction network, specifically including:

[0082] Use the feature extraction network f(·) to extract the background layer features f I and rain layer characteristics f R ,in,

[0083] f I =f(I k+1 ), f R =f(R k+1 );

[0084] Maximize the background layer feature f I and rain layer characteristics f R The difference between and is used to calculate the contrast loss value:

[0085]

[0086] Among them, L contrast is the contrast loss value, f I is the background layer feature, f R is the rain layer feature, s(·) is the similarity function, τ is the temperature coefficient, and N is the number of negative samples.

[0087] In this solution, the total loss value is calculated based on the high-dimensional features, the original rainy image, and the contrast loss value, specifically including:

[0088] Calculate the model loss value:

[0089]

[0090] Among them, L model is the model loss value, is the reconstruction error term, J is the original rainy image, I is the background layer to be optimized, R is the rain layer to be optimized, is the regularization loss of the background layer, ‖R‖1 is the sparse loss of the rain layer;

[0091] Calculate the total loss value:

[0092] L total =L model +λL contrast ;

[0093] Among them, L total is the total loss value, L model is the model loss value, L contrast is the contrast loss value, and λ is the loss weight.

[0094] A third aspect of the present invention provides a contrastive learning image deraining method, which is applied to a deraining model trained by any of the contrastive learning image deraining model training methods, wherein the method comprises the following steps:

[0095] Get a test image;

[0096] Inputting the test image into a trained deraining model to obtain a processing result, wherein the processing result includes a derained image and a rain layer;

[0097] The processing results are extracted and differentiated and outputted, wherein, based on the differentiation of result types, the derained image result must be outputted, and the rain layer result can be outputted optionally.

[0098] A fourth aspect of the present invention provides a contrastive learning image rain removal system, comprising a memory and a processor, wherein the memory comprises a contrastive learning image rain removal method program, and when the contrastive learning image rain removal method program is executed by the processor, the following steps are implemented:

[0099] Get a test image;

[0100] Inputting the test image into a trained deraining model to obtain a processing result, wherein the processing result includes a derained image and a rain layer;

[0101] The processing results are extracted and differentiated and outputted, wherein, based on the differentiation of result types, the derained image result must be outputted, and the rain layer result can be outputted optionally.

[0102] A fifth aspect of the present invention provides a computer-readable storage medium, which includes a comparative learning image deraining model training method program. When the comparative learning image deraining model training method program is executed by a processor, it implements the steps of any one of the comparative learning image deraining model training methods. The computer-readable storage medium also includes a comparative learning image deraining method program. When the comparative learning image deraining method program is executed by a processor, it implements the steps of any one of the comparative learning image deraining method methods.

[0103] The present invention discloses a contrastive learning image rain removal model training method, rain removal method, and storage medium. By alternately iteratively solving the rain layer and the image layer and applying contrast loss constraints, the method can better distinguish the rain layer and the image layer, ultimately achieving separation of the rain layer and the image layer. The specific beneficial effects are as follows:

[0104] 1. A dual-constraint mechanism, combining sparse optimization and contrastive learning, constrains rain streaks and background simultaneously at the image level and in feature space, significantly improving separation accuracy.

[0105] 2. Adaptive optimization, in which contrastive loss dynamically guides sparse coding updates and utilizes hierarchical feature evolution information to enhance the model's adaptability.

[0106] 3. Strong detail preservation capability. Background layer regularization preserves sharp edges, while rain layer thinning suppresses noise, effectively avoiding the loss of background details.

[0107] 4. Efficient iterative separation: alternating iterative optimization ensures that the rain streaks and background layer gradually approach their true distribution with each update, ultimately achieving precise separation.

[0108] 5. Wide applicability: The model's network module design is flexible (such as U-Net, residual dense network, etc.), which can adapt to the needs of different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] Figure 1 A diagram showing the steps of a comparative learning image deraining model training method according to the present invention is shown;

[0110] Figure 2 A flowchart of a comparative learning image deraining model training method of the present invention is shown;

[0111] Figure 3 A block diagram of a contrastive learning image deraining model training system according to the present invention is shown;

[0112] Figure 4 A step diagram of a comparative learning image deraining method according to the present invention is shown;

[0113] Figure 5 A block diagram of a contrastive learning image deraining system according to the present invention is shown. DETAILED DESCRIPTION

[0114] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0115] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0116] Current mainstream image deraining methods primarily employ end-to-end supervised learning strategies, training the network by constructing a pixel-level mapping between rainy and rain-free images. These methods typically directly impose constraints on the derained background layer, aiming to make it as close to a true rain-free image as possible. Furthermore, existing methods rely on single-dimensional constraints (such as pixel-level loss), making it difficult to fully capture the complex interactions between rain streaks and the background. This results in suboptimal deraining performance, particularly in preserving background detail and separating rain streaks. These methods suffer from numerous shortcomings: A single constraint makes it difficult to distinguish the complex features of rain streaks and background; traditional methods are unable to separate rain streaks and background in feature space, resulting in a loss of background detail or residual rain streaks; and they lack adaptive learning mechanisms, making it impossible to dynamically optimize the feature distributions of rain streaks and background.

[0117] Therefore, in order to effectively separate rain streaks and background layers and avoid the loss of background details, this application designs a model-guided contrast learning image deraining method for rain images taken outdoors, thereby improving image quality and the reliability of the visual system. The distinguishability of rain streaks and background in the feature space is enhanced through contrast learning, and adaptive iterative optimization is achieved by combining sparse optimization and contrast learning. Among them, a model-guided contrast learning image deraining method is adopted. On the one hand, by modeling the rain layer and background layer separately, it is ensured that the decomposed image satisfies the rain superposition model, and the network is used to adaptively learn and constrain the structural characteristics of the rain layer and background layer; on the other hand, a contrast loss function is introduced, and the network parameters are continuously optimized through iteration to make the feature distributions of the rain layer and the background layer significantly separable, thereby improving the separation accuracy.

[0118] Specifically, Figure 1 A flowchart of a comparative learning image deraining model training method of the present application is shown.

[0119] like Figure 1 As shown, the present application discloses a contrast learning image deraining model training method, comprising the following steps:

[0120] S102, obtaining an original rainy image and inputting it into an initialized rain removal model to obtain initial data, wherein the initial data includes an initial background layer and an initial rain layer;

[0121] S104, alternately fixing the background layer and the rain layer based on the initial data to iteratively optimize the rain removal model, wherein the rain removal model is composed of the background layer and the rain layer;

[0122] S106, extracting high-dimensional features of the iteratively optimized background layer and rain layer based on the feature extraction network and calculating a contrast loss value, wherein the high-dimensional features include background layer features and rain layer features;

[0123] S108, calculating a total loss value based on the high-dimensional features, the original rainy image, and the contrast loss value, and back-propagating the model parameters based on the model loss value;

[0124] S110, when the number of iterations reaches a preset value or the total loss value converges, the iteration is stopped, and the final derained image and rain layer are output.

[0125] It should be noted that, in this embodiment, Figure 2 As shown in the figure, it is a flowchart of contrastive learning image deraining model training, in which a rainy image is first input, and in each iteration, the rain layer extractor E is first fixed. R , using the background layer extractor E I Update the background image; then fix the background layer extractor E I , use the rain layer extractor to update the rain layer image E R , the output rain layer and the output background layer are obtained in sequence through two extractors, among which the background layer extractor E I or Rain Layer Extractor E R They are all composed of corresponding optimization modules and network models.

[0126] Specifically, the original rainy image is input into the initialized rain removal model to obtain the initial data, which includes the initial background layer and the initial rain layer, that is, the rainy image J i Input Figure 2 In the network shown, the rain layer output of the first iteration is obtained through the background layer extractor and the rain layer extractor. and background layer in, Among them E I and E R The second parameter is the rain layer and background layer corresponding to the iteration. Since the background layer extractor has not yet generated a rain layer in the first iteration, "0" is used instead.

[0127] Furthermore, the layer is obtained again and input into the background layer extractor and the rain layer extractor again to guide better extraction of the rain layer and the background layer. Taking the kth iteration as an example, it is input into the network module of the background layer extractor after optimization using the contrast loss for the next iteration. Finally, the contrast loss is calculated for the two output image layers and the network parameters of the background layer extractor are optimized through back propagation. Then it is iterated in sequence. When the number of iterations reaches the preset value or the total loss value converges, the iteration is stopped and the final derained image and rain layer are output to achieve the final effective deraining.

[0128] According to an embodiment of the present invention, the objective function of the rain removal model is as follows:

[0129]

[0130] in, is the reconstruction error term, which is used to decompose the input original rainy image into the background layer I and the rain layer R. and It is the regularization prior applied to the background layer and rain layer of the image, J is the original rainy image, I is the background layer to be optimized, and R is the rain layer to be optimized.

[0131] It should be noted that, in this embodiment, the reconstruction error term To reconstruct the error term, the input image J is decomposed into a linear combination of the background layer and the rain layer, where I is the background layer to be optimized and R is the rain layer to be optimized. Furthermore, the background layer constraint term The background layer is kept sharp and smooth, while the rain layer is constrained It is used to suppress the noise of the rain layer.

[0132] According to an embodiment of the present invention, fixing the rain layer and iteratively optimizing the background layer specifically includes:

[0133] Subproblem of calculating the background layer:

[0134]

[0135] Among them, I k+1 is the updated background layer, I is the background layer to be optimized, R k is the current rain layer, J is the original rainy image, is the regularization prior applied to the image background layer;

[0136] Calculate the background layer residual gradient:

[0137] r I =I k +ρ(JI k -R k );

[0138]

[0139] Among them, r I is the background layer residual gradient, ρ is the control step size, I k is the current background layer, R k is the current rain layer, J is the original rainy image, I k+1 is the updated background layer, I is the background layer to be optimized, is the regularization prior applied to the image background layer;

[0140] The background layer after iterative update is:

[0141]

[0142] Among them, I k+1 For the updated background layer, is the background layer extraction function, r I is the residual gradient of the background layer.

[0143] It should be noted that, in this embodiment, the rain layer is fixed and the background layer is optimized. After calculation using the formula described in this embodiment, the background layer can be enhanced, wherein the background layer residual gradient r I Input to the background layer extractor (such as U-Net or residual dense network) to extract the background layer to obtain the updated background layer I k+1 .

[0144] According to an embodiment of the present invention, the background layer is fixed and the rain layer is iteratively optimized, specifically including:

[0145] Subproblem of computing the rain layer:

[0146]

[0147] Among them, R k+1 For the updated rain layer, I k+1 is the updated background layer, R is the rain layer to be optimized, and J is the original rainy image. is the regularization prior applied to the rain layer of the image;

[0148] Calculate the residual gradient of the rain layer:

[0149] r R =R k +ρ(JI k+1 -R k );

[0150]

[0151] Among them, r R is the residual gradient of the rain layer, R k is the current rain layer, ρ is the control step size, R k+1 is the updated rain layer, R k is the current rain layer, J is the original rainy image, I k+1 is the updated background layer, R is the rain layer to be optimized, is the regularization prior applied to the rain layer of the image;

[0152] The rain layer after iterative update is:

[0153]

[0154] Among them, R k+1 For the updated rain layer, is the rain layer extraction function, rR is the residual gradient of the rain layer.

[0155] It should be noted that, in this embodiment, the background layer is fixed and the rain layer is optimized. After calculation using the formula described in this embodiment, the rain layer can be enhanced, wherein the background layer residual gradient r R Input to the rain layer extractor (such as lightweight CNN) to extract the rain layer and obtain the updated rain layer R k+1 .

[0156] According to an embodiment of the present invention, the high-dimensional features of the background layer and the rain layer after iterative optimization are extracted based on the feature extraction network, and the contrast loss value is calculated, specifically including:

[0157] Use the feature extraction network f(·) to extract the background layer features f I and rain layer characteristics f R ,in,

[0158] f I =f(I k+1 ), f R =f(R k+1 );

[0159] Maximize the background layer feature f I and rain layer characteristics f R The difference between and is used to calculate the contrast loss value:

[0160]

[0161] Among them, L contrast is the contrast loss value, f I is the background layer feature, f R is the rain layer feature, s(·) is the similarity function, τ is the temperature coefficient, and N is the number of negative samples.

[0162] It should be noted that, in this embodiment, for the contrast learning constraint, the input value includes the updated background layer I k+1 and Rain Layer R k+1 , use the feature extraction network f(·) to extract the background layer features f I and rain layer characteristics f R , maximize the background layer feature f I and rain layer characteristics f R The difference between the same features is minimized to calculate the contrast loss value L contrast , where the similarity function s(·) can adopt the cosine similarity function when applied.

[0163] According to an embodiment of the present invention, calculating the total loss value based on the high-dimensional features, the original rainy image, and the contrast loss value specifically includes:

[0164] Calculate the model loss value:

[0165]

[0166] Among them, L model is the model loss value, is the reconstruction error term, J is the original rainy image, I is the background layer to be optimized, R is the rain layer to be optimized, is the regularization loss of the background layer, ‖r‖1 is the sparse loss of the rain layer;

[0167] Calculate the total loss value:

[0168] L total =L model +λL contrast ;

[0169] Among them, L total is the total loss value, L model is the model loss value, L contrast is the contrast loss value, and λ is the loss weight.

[0170] It should be noted that, in this embodiment, the model loss value consists of three parts, which respectively constrain the image reconstruction consistency, background layer structure and rain layer sparsity. Among them, the reconstruction error term The purpose is to force the background layer after rain removal to accurately reconstruct the original rainy image, so as to ensure the physical rationality of the decomposition and avoid the loss of background or rain layer information, while the background layer is regularly lost. It is to suppress the noise and non-real texture of the background layer, where corresponds to the gradient of the background layer, ‖·‖1 represents the L1 norm of the gradient amplitude. Correspondingly, the rain layer sparse loss ‖R‖1 is used to suppress non-rain noise (such as fog interference) in the rain layer, and ‖R‖1 corresponds to the L1 norm of the rain layer.

[0171] Furthermore, the total loss value L total is the model loss value L model Compared with the loss value L contrast The sum of λ and λ, where λ is the loss weight. The larger λ is, the more the model focuses on the feature separation of rain streaks and background, which can improve the robustness of rain removal. Correspondingly, λ∈[0.1,1]. Furthermore, the model parameters are updated based on the back propagation of the model loss value. The parameter update is achieved through the back propagation algorithm, and its core steps include gradient calculation, loss back propagation and optimizer update. These are conventional technical means that can be selected by those skilled in the art and will not be elaborated here.

[0172] Figure 3 A block diagram of a contrastive learning image deraining model training system of the present invention is shown.

[0173] like Figure 3 As shown, the present invention discloses a contrastive learning image deraining model training system, including a memory and a processor. The memory includes a contrastive learning image deraining model training method program. When the contrastive learning image deraining model training method program is executed by the processor, the following steps are implemented:

[0174] Obtaining an original rainy image and inputting it into an initialized rain removal model to obtain initial data, wherein the initial data includes an initial background layer and an initial rain layer;

[0175] The background layer and the rain layer are alternately fixed based on the initial data to iteratively optimize the rain removal model, wherein the rain removal model is composed of the background layer and the rain layer,

[0176] Extracting high-dimensional features of the iteratively optimized background layer and rain layer based on a feature extraction network and calculating a contrast loss value, wherein the high-dimensional features include background layer features and rain layer features;

[0177] Calculating a total loss value based on the high-dimensional features, the original rainy image, and the contrast loss value, and updating model parameters based on the model loss value through backpropagation;

[0178] When the number of iterations reaches the preset value or the model loss value converges, the iteration stops and the final derained image and rain layer are output.

[0179] It should be noted that, in this embodiment, the technical solution and detailed implementation are consistent with the above-mentioned embodiment of the comparative learning image deraining model training method, and will not be repeated here.

[0180] Furthermore, Figure 4 A flowchart of a comparative learning image deraining model training method of the present application is shown.

[0181] like Figure 4 As shown, the present application discloses a contrast learning image deraining model training method, comprising the following steps:

[0182] S402, acquiring a test image;

[0183] S404: Input the test image into the trained rain removal model to obtain a processing result, wherein the processing result includes a rain removal image and a rain layer;

[0184] S406: extract the processing results and output them in a differentiated manner. Based on the result type, the derained image result must be output, and the rain layer result can be output optionally.

[0185] It should be noted that, in this embodiment, after the iterative optimization training of the deraining model is completed, it can be used for testing. Accordingly, a test image is obtained and input into the trained deraining model to obtain the corresponding processing result, including the deraining image and the rain layer, and then the processing result is extracted and output. Among them, there is a differential output when outputting, that is, the deraining image is necessarily output, while the rain layer can be selectively output according to user needs.

[0186] Figure 5 A block diagram of a contrastive learning image deraining system according to the present invention is shown.

[0187] like Figure 5 As shown, the present invention discloses a contrastive learning image deraining system, comprising a memory and a processor. The memory comprises a contrastive learning image deraining method program. When the contrastive learning image deraining method program is executed by the processor, the following steps are implemented:

[0188] Get a test image;

[0189] Inputting the test image into a trained deraining model to obtain a processing result, wherein the processing result includes a derained image and a rain layer;

[0190] The processing results are extracted and differentiated and outputted, wherein, based on the differentiation of result types, the derained image result must be outputted, and the rain layer result can be outputted optionally.

[0191] It should be noted that, in this embodiment, the technical solution and detailed implementation are consistent with the above-mentioned embodiment of the comparative learning image deraining method, and are not described in detail here.

[0192] A fifth aspect of the present invention provides a computer-readable storage medium, which includes a comparative learning image deraining model training method program. When the comparative learning image deraining model training method program is executed by a processor, it implements the steps of any one of the comparative learning image deraining model training methods. The computer-readable storage medium also includes a comparative learning image deraining method program. When the comparative learning image deraining method program is executed by a processor, it implements the steps of any one of the comparative learning image deraining method methods.

[0193] The present invention discloses a contrastive learning image rain removal model training method, rain removal method and storage medium. By alternately iteratively solving the rain layer and the image layer and performing contrast loss constraints, the rain layer and the image layer can be better distinguished and ultimately separated.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0195] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0196] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0197] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0198] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A contrastive learning image deraining model training method, characterized by: The following steps are involved: Obtaining an original rainy image and inputting it into an initialized rain removal model to obtain initial data, wherein the initial data includes an initial background layer and an initial rain layer; Alternately fixing the background layer and the rain layer based on the initial data to iteratively optimize the rain removal model, the rain removal model consisting of the background layer and the rain layer, in, Extracting high-dimensional features of the iteratively optimized background layer and rain layer based on a feature extraction network and calculating a contrast loss value, wherein the high-dimensional features include background layer features and rain layer features; Calculating a total loss value based on the high-dimensional features, the original rainy image, and the contrast loss value, and updating model parameters based on the model loss value through backpropagation; When the number of iterations reaches the preset value or the total loss value converges, the iteration stops and the final derained image and rain layer are output.

2. The contrastive learning image deraining model training method according to claim 1, characterized in that: The objective function of the rain removal model is as follows: in, is the reconstruction error term, which is used to decompose the input original rainy image into the background layer I and the rain layer R. and It is the regularization prior applied to the background layer and rain layer of the image, J is the original rainy image, I is the background layer to be optimized, and R is the rain layer to be optimized.

3. The contrastive learning image deraining model training method according to claim 2, characterized in that: Fixed the rain layer and iteratively optimized the background layer, including: Subproblem of calculating the background layer: Among them, I k+1 is the updated background layer, I is the background layer to be optimized, R k is the current rain layer, J is the original rainy image, is the regularization prior applied to the image background layer; Calculate the background layer residual gradient: r I =I k +ρ(J-I k -R k ); Among them, r I is the background layer residual gradient, ρ is the control step size, I k is the current background layer, R k is the current rain layer, J is the original rainy image, I k+1 is the updated background layer, I is the background layer to be optimized, is the regularization prior applied to the image background layer; The background layer after iterative update is: Among them, I k+1 For the updated background layer, is the background layer extraction function, r I is the residual gradient of the background layer.

4. The contrastive learning image deraining model training method according to claim 3, characterized in that: Fixed the background layer and iteratively optimized the rain layer, including: Subproblem of computing the rain layer: Among them, R k+1 For the updated rain layer, I k+1 is the updated background layer, R is the rain layer to be optimized, and J is the original rainy image. is the regularization prior applied to the rain layer of the image; Calculate the residual gradient of the rain layer: r R =R k +ρ(J-I k+1 -R k ); Among them, r R is the residual gradient of the rain layer, R k is the current rain layer, ρ is the control step size, R k+1 is the updated rain layer, R k is the current rain layer, J is the original rainy image, I k+1 is the updated background layer, R is the rain layer to be optimized, is the regularization prior applied to the rain layer of the image; The rain layer after iterative update is: Among them, R k+1 For the updated rain layer, is the rain layer extraction function, r R is the residual gradient of the rain layer.

5. The contrastive learning image deraining model training method according to claim 4, characterized in that: The high-dimensional features of the iteratively optimized background layer and rain layer are extracted based on the feature extraction network to calculate the contrast loss value, specifically including: Use the feature extraction network f(·) to extract the background layer features f I and rain layer characteristics f R ,in, f I =f(I k+1 ),f R =f(R k+1 ); Maximize the background layer feature f I and rain layer characteristics f R The difference between and is used to calculate the contrast loss value: Among them, L contrast is the contrast loss value, f I is the background layer feature, f R is the rain layer feature, s(·) is the similarity function, τ is the temperature coefficient, and N is the number of negative samples.

6. The contrastive learning image deraining model training method according to claim 5, characterized in that: Calculating a total loss value based on the high-dimensional feature, the original rainy image, and the contrast loss value specifically includes: Calculate the model loss value: Among them, L model is the model loss value, is the reconstruction error term, J is the original rainy image, I is the background layer to be optimized, R is the rain layer to be optimized, is the regularization loss of the background layer, ‖R‖1 is the sparse loss of the rain layer; Calculate the total loss value: THE total =L model +λL contrast ; Among them, L total is the total loss value, L model is the model loss value, L contrast is the contrast loss value, and λ is the loss weight.

7. A contrastive learning image deraining model training system, characterized by: The system comprises a memory and a processor, wherein the memory comprises a contrastive learning image deraining model training method program, and when the contrastive learning image deraining model training method program is executed by the processor, the following steps are implemented: Obtaining an original rainy image and inputting it into an initialized rain removal model to obtain initial data, wherein the initial data includes an initial background layer and an initial rain layer; Alternately fixing the background layer and the rain layer based on the initial data to iteratively optimize the rain removal model, the rain removal model consisting of the background layer and the rain layer, in, Extracting high-dimensional features of the iteratively optimized background layer and rain layer based on a feature extraction network and calculating a contrast loss value, wherein the high-dimensional features include background layer features and rain layer features; Calculating a total loss value based on the high-dimensional features, the original rainy image, and the contrast loss value, and updating model parameters based on the model loss value through backpropagation; When the number of iterations reaches the preset value or the model loss value converges, the iteration stops and the final derained image and rain layer are output.

8. A contrastive learning image deraining method, characterized in that: A deraining model obtained by training the contrastive learning image deraining model training method according to any one of claims 1 to 6, wherein the method comprises the following steps: Get a test image; Inputting the test image into a trained deraining model to obtain a processing result, wherein the processing result includes a derained image and a rain layer; The processing results are extracted and differentiated and outputted, wherein, based on the differentiation of result types, the derained image result must be outputted, and the rain layer result can be outputted optionally.

9. A contrastive learning image deraining system, characterized in that: The system comprises a memory and a processor, wherein the memory comprises a program for a comparative learning image deraining method, and when the program is executed by the processor, the following steps are implemented: Get a test image; Inputting the test image into a trained deraining model to obtain a processing result, wherein the processing result includes a derained image and a rain layer; The processing results are extracted and differentiated and outputted, wherein, based on the differentiation of result types, the derained image result must be outputted, and the rain layer result can be outputted optionally.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a contrastive learning image deraining model training method program. When the contrastive learning image deraining model training method program is executed by a processor, the steps of a contrastive learning image deraining model training method according to any one of claims 1 to 6 are implemented. The computer-readable storage medium also includes a contrastive learning image deraining method program. When the contrastive learning image deraining method program is executed by a processor, the steps of a contrastive learning image deraining method according to claim 8 are implemented.

Images