Self-adaptive image super-resolution reconstruction method and system in original-domain-free single sample domain

Abstract

The invention discloses an original-domain-free single-sample-domain adaptive image super-resolution reconstruction method and system. The reconstruction method comprises the following steps: 1, adjusting the learning rate of each network layer in a target domain network by using a target domain low-resolution image and an original network; 2, initializing a target domain network and a teacher network by adopting a target domain low-resolution image and an original domain network; 3, adjusting the learning rate of each network layer in the target domain network by using the target domain low-resolution image and the teacher network; 4, performing field adaptive training on the target domain network by using the teacher network; and 5, obtaining a reconstructed super-resolution image by using the trained target domain network. According to the method, domain self-adaption can be carried out on the target domain network under the conditions that an original domain image is not accessed and only a single target domain low-resolution image exists, the negative influence of a domain gap is reduced, and the performance of the super-resolution network in the target domain is effectively improved.

Description

Technical Field

[0001] The invention relates to a method and system for self-adaptive image super-resolution reconstruction without original domain single-sample domain, belonging to the technical field of image processing. Background Art

[0002] Super-resolution has a very broad range of applications, such as sharpening old videos, remote sensing imaging, and can even be used as a preprocessing method for downstream tasks (target detection, semantic segmentation).

[0003] However, super-resolution networks usually face the problem of domain gap. Due to differences in imaging equipment, lighting environment, camera shake, etc., the distribution of training data (original domain) is inconsistent with the feature distribution of actual application scenarios (target domain), which will cause the super-resolution network with good training effect in the original domain to have serious performance degradation in the target domain. In addition, due to privacy clauses, the training data of the original domain network is usually confidential within the company and is prohibited from flowing out to downstream tasks. Therefore, domain adaptation methods without original domain samples have emerged. They use teacher networks to generate pseudo labels to achieve domain adaptation without accessing the original domain. However, in some real-world scenarios, for example, pictures taken and uploaded by users usually contain only a small number of samples, or even only a single sample. Directly using the above-mentioned domain adaptation methods without original domain samples in the current scenario will face the risk of mode collapse, which will seriously attenuate the performance of the network. Summary of the invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a domain-adaptive image super-resolution reconstruction method without an original domain image and with only a single target domain low-resolution image. This method can perform domain adaptation on the target domain network without accessing the original domain image and with only a single target domain low-resolution image, thereby reducing the negative impact of the domain gap and effectively improving the performance of the super-resolution network in the target domain.

[0005] In order to solve the above technical problems, the present invention discloses a method for self-adaptive image super-resolution reconstruction without original domain single-sample, comprising the steps of:

[0006] S1, using the target domain low-resolution image and the original network to adjust the learning rate of each network layer in the target domain network;

[0007] The original domain network has completed pre-training; the target domain network has completed pre-initialization;

[0008] The structure of the original domain network is a MambaIR network, including a first convolutional layer, a first deep feature extraction network, and a first upsampling layer; the first convolutional layer is used to extract shallow image features of the original domain network input image, the first deep feature extraction network is used to extract deep image features from shallow image features, and the first deep feature extraction network is composed of a plurality of RSSGs cascaded in sequence; the first upsampling layer is used to generate a reconstructed super-resolution image according to shallow image features and deep image features;

[0009] The target domain network includes a second convolutional layer, a semantic perception module, a second deep feature extraction network, and a second upsampling layer; the second convolutional layer is used to extract shallow image features of the input spliced ​​low-resolution image, and the semantic perception module is used to generate semantically guided image features according to the input main semantic information and shallow image features; the second deep feature extraction network is used to extract deep image features in the semantically guided image features, and the second upsampling layer is used to generate a reconstructed super-resolution image according to the shallow image features and the deep image features;

[0010] The first convolutional layer has the same structure as the second convolutional layer; the first deep feature extraction network has the same structure as the second deep feature extraction network; the first upsampling layer has the same structure as the second upsampling layer;

[0011] The semantic perception module expands the incoming shallow image features in four directions: horizontal, vertical, reverse horizontal and reverse vertical, to generate four expanded image features; at the same time, the incoming main semantic information is copied in the channel dimension and the quantity dimension, aligned with the four expanded image features, to generate copied semantic information; the copied semantic information is spliced ​​before the four expanded image features to generate image features to be semantically guided; the image features to be semantically guided are input into the SSM module to generate scanned image features; the scanned image features are split, the features at the corresponding positions of the main semantic information are cropped, and guided image features are generated; the guided image features are added in the quantity dimension to generate merged image features; the merged image features are subjected to shape restoration processing to obtain semantically guided image features, and the size of the semantically guided image features is consistent with the size of the shallow image features passed to the semantic perception module;

[0012] S2. Initialize the target domain network and the teacher network using the target domain low-resolution image and the original domain network; the teacher network has the same structure as the target domain network, and the teacher network has been pre-initialized;

[0013] S3, using the target domain low-resolution image and the teacher network to adjust the learning rate of each network layer in the target domain network;

[0014] S4, use the teacher network to perform domain adaptive training on the target domain network;

[0015] S5. Using the trained target domain network, a reconstructed target domain super-resolution image is obtained. The steps are as follows:

[0016] The target domain low-resolution image is split into multiple image blocks and the corresponding position mask is generated; the target domain low-resolution image and the position mask are input into the Alpha-CLIP network to generate the corresponding semantic prior information; each split image block together with the semantic prior information is input into the trained target domain network to generate multiple split target domain super-resolution images; the multiple split target domain super-resolution images are spliced ​​in the spatial dimension to generate a reconstructed target domain super-resolution image.

[0017] Specifically, the pre-training of the original domain network includes:

[0018] Inputting a low-resolution image in the original domain data set into the original domain network, and outputting an original domain super-resolution image; calculating a first loss function value according to the original domain super-resolution image and the high-resolution image in the original domain data set, and training the original domain network by minimizing the first loss function value to obtain a pre-trained original domain network;

[0019] The original domain dataset includes a plurality of high-resolution images and low-resolution images corresponding to the high-resolution images.

[0020] The pre-initialization of the target domain network and the teacher network includes:

[0021] The parameters of the second convolutional layer, the second deep feature extraction network, and the second upsampling layer of the target domain network and the teacher network are respectively set to the parameters of the first convolutional layer, the first deep feature extraction network, and the first upsampling layer of the pre-trained original domain network;

[0022] The parameters of the semantic perception modules in the target domain network and the teacher network are randomly initialized.

[0023] Specifically, the method of adjusting the learning rate of each network layer in the target domain network by using the target domain low-resolution image and the original network includes:

[0024] S11. Initialize the number of cropped samples N p , the number of domain specialization layers N s , learning rate increase multiple P i , learning rate reduction ratio P d ;

[0025] S12: Perform N on the low-resolution image in the target domain p Random cropping and prior information acquisition operations are performed to generate N p cropped low-resolution images and corresponding semantic information; pThe cropped low-resolution image is copied 6 times and geometrically enhanced to generate 6*N p geometrically enhanced low-resolution images; 6*N p A geometrically enhanced low-resolution image is input into the original domain network to generate 6*N p super-resolution pseudo-label images; 6*N p The super-resolution pseudo-label images are processed for position restoration respectively to generate 6*N p Position-aligned super-resolution pseudo-label images; 6*N p The super-resolution pseudo-label images aligned at each position are averaged to generate N p pseudo-label images; p The cropped low-resolution images and the corresponding semantic information are input into the target domain network for super-resolution reasoning to generate N p Super-resolution images;

[0026] Calculate N p super-resolution images and N p The L1 loss between the pseudo-label images is calculated, and the gradient is returned to save the gradient of each network layer in the target domain network; the average gradient of each network layer in the target domain network is calculated to obtain the average gradient of each network layer in the target domain network;

[0027] Traverse each network layer of the target domain network. If the average gradient of the current network layer is in the top N of the average gradient size of all network layers of the target domain network, s Among them, the learning rate of the parameters of the current network layer increases by P i times, otherwise, the learning rate of the parameters of the current network layer is reduced by P d .

[0028] Specifically, the random cropping and prior information acquisition operations are:

[0029] The image to be processed is randomly cropped into a cropped image with a pixel width of w and a pixel height of h, and a corresponding position mask is generated at the same time; the position mask is consistent with the size of the image to be processed and is used to indicate the position of the cropped image in the image to be processed;

[0030] The position mask and the image to be processed are input into the Alpha-CLIP network to generate semantic information corresponding to the cropped image.

[0031] Specifically, the target domain low-resolution image and the original domain network are used to initialize the target domain network and the teacher network, including the steps of:

[0032] S21, generating a main low-resolution image LR based on the target domain low-resolution image p , secondary low-resolution image LR aand the main semantic information S p , specifically including the steps:

[0033] S211, perform two random cropping and prior information acquisition operations on the target domain low-resolution image to obtain two cropped low-resolution images and And the corresponding semantic information S a and S b ;

[0034] S212, for cropping low-resolution images and Perform random fuzzy data enhancement processing to generate a fuzzy enhanced cropped low-resolution image and

[0035] S213, Random Assignment and One of them is used as the main low-resolution image LR p , the other is a low-resolution image LR a ; At the same time, the semantic information corresponding to the main low-resolution image is designated as the main semantic information S p ;

[0036] S22, using the target domain network to LR the main low-resolution image p , secondary low-resolution image LR a and the main semantic information S p Processing is performed to generate the main super-resolution image SR p ; Use the original domain network to LR the main low-resolution image p Process and generate pseudo label PL S ;

[0037] Generating a primary super-resolution image SR p The specific steps include:

[0038] S2211, the main low-resolution image LR p The secondary low-resolution image is spliced ​​in a random order to generate a spliced ​​low-resolution image CLR;

[0039] S2212, input the spliced ​​low-resolution image CLR into the second convolutional layer in the target domain network, and input the main semantic information S p The semantic perception module in the target domain network is input, and the second upsampling layer in the target domain network outputs the spliced ​​super-resolution image CSR;

[0040] S2213, split the spliced ​​super-resolution image CSR, retaining the main semantic information S pThe corresponding super-resolution pixel area is used as the main super-resolution image SR p ;

[0041] The pseudo label PL is generated S Specifically include:

[0042] S2221, the main low-resolution image LR p Duplicate 6 copies and perform geometric enhancement to generate 6 geometrically enhanced low-resolution images

[0043] Input 6 geometrically enhanced low-resolution images into the original domain network to generate 6 super-resolution images

[0044] S2222, performing position restoration processing on the six super-resolution images respectively to generate six position-aligned super-resolution images;

[0045] S2223, calculate the average value of the super-resolution images aligned at the 6 positions and generate a pseudo label PL S ;

[0046] S23, according to the pseudo label PL S and the main super-resolution image SR p Calculating a second loss function value, and optimizing the target domain network by minimizing the second loss function value;

[0047] S24. Synchronously update the teacher network parameters using exponential averaging:

[0048]

[0049] Represent the teacher network parameters before and after updating, is the parameter of the target domain network, and γ represents the preset retention coefficient.

[0050] Specifically, the random fuzzy data enhancement process is:

[0051] The image P to be processed is downsampled and upsampled twice to generate a forged image P fake ;

[0052] Generate a random mask Mask. The size of the random mask Mask is consistent with the size of the image P to be processed. The random mask Mask is divided into a grid with a pixel width of w1 and a pixel height of h1. The pixel value in each grid is set to 1 with a probability of 40%;

[0053] The image BAug(P) after random fuzzy data enhancement is:

[0054] BAug(P)=Mask*P+Rev(Mask)*P fake

[0055] Rev(Mask) means to perform a zero-one flip operation on the Mask, that is, to swap each pixel value in the random mask, the pixel value of 1 is swapped to 0, and the pixel value of 0 is swapped to 1.

[0056] Specifically, the method of adjusting the learning rate of each network layer in the target domain network by using the target domain low-resolution image and the teacher network includes:

[0057] S31, initialize the number of cropped samples N p , the number of domain specialization layers N s , learning rate increase multiple P i , learning rate reduction ratio P d ;

[0058] S32, perform N on the target domain low-resolution image p Random cropping and prior information acquisition operations are performed to generate N p cropped low-resolution images and corresponding position masks; p cropped low-resolution images and N p Each position mask is copied 6 times and geometrically enhanced to generate 6*N p geometrically enhanced low-resolution images and 6*N p geometric enhancement position masks; copy the target domain low-resolution image 6*N p times, and performs geometric enhancement processing to generate 6*N p The number of aligned target domain low-resolution images; 6*N p The geometrically enhanced position mask together with 6*N p The number of aligned target domain low-resolution images is sent to the Alpha-CLIP network to generate 6*N p semantic information; 6*N p geometrically enhanced low-resolution images together with 6*N p The semantic information is input into the teacher network together to generate 6*N p super-resolution pseudo-label images; 6*N p Super-resolution pseudo-label images are processed for position restoration to generate 6*N p Position-aligned super-resolution pseudo-label images; 6*N p The super-resolution pseudo-label images aligned at each position are averaged to generate N p pseudo-label images; p The cropped low-resolution images and their corresponding semantic information are input into the target domain network for super-resolution reasoning to generate Np Super-resolution images;

[0059] Calculate N p super-resolution images and N p The L1 loss between the pseudo-label images is calculated, and the gradient is returned to save the gradient of each network layer in the target domain network; the average gradient of each network layer in the target domain network is calculated to obtain the average gradient of each network layer in the target domain network;

[0060] Traverse each network layer of the target domain network. If the average gradient of the current network layer is in the top N of the average gradient size of all network layers of the target domain network, s Among them, the learning rate of the parameters of the current network layer increases by P i times, otherwise, the learning rate of the parameters of the current network layer is reduced by P d .

[0061] Specifically, the method of using the teacher network to perform domain adaptive training on the target domain network includes the following steps:

[0062] S41, generating a main low-resolution image LR according to the target domain low-resolution image p , secondary low-resolution image LR a and the main semantic information S p ;

[0063] LR of the main low-resolution image using the target domain network p , secondary low-resolution image LR a and the main semantic information S p Processing is performed to generate the main super-resolution image SR p ;

[0064] S42, use the teacher network to process the low-resolution image of the target domain and generate a pseudo label PL Te , specifically including:

[0065] The target domain low-resolution image is cropped, and the cropped position and size are consistent with the main low-resolution image to generate a cropped low-resolution image and a corresponding position mask; the cropped low-resolution image, the corresponding position mask and the target domain low-resolution image are copied 6 times respectively, and geometrically enhanced to generate 6 geometrically enhanced cropped target domain low-resolution images 6 geometry-enhanced position masks and 6 geometrically enhanced target domain low-resolution images The 6 geometrically enhanced position masks and 6 geometrically enhanced target domain low-resolution images are input into the Alpha-CLIP network to generate 6 semantic prior information S Te ; 6 geometrically enhanced cropped target domain low-resolution images and 6 semantic prior information S Te Send it to the teacher network to generate 6 super-resolution images The six super-resolution images are processed for position restoration to generate a super-resolution image with position alignment; the average value of the super-resolution images with position alignment is calculated to generate a pseudo label PL Te ;

[0066] S43, according to the pseudo label PL Te and the main super-resolution image SR p Calculating a third loss function value, and optimizing the target domain network by minimizing the third loss function value;

[0067] S44. Synchronously update the teacher network parameters using exponential averaging:

[0068]

[0069] Represent the teacher network parameters before and after updating, is the parameter of the target domain network, and γ represents the preset retention coefficient.

[0070] On the other hand, the present invention also discloses a non-original domain single-domain adaptive image super-resolution reconstruction system, comprising:

[0071] The target domain network learning rate adjustment first module 100 is used to adjust the learning rate of each network layer in the target domain network using the target domain low-resolution image and the original network;

[0072] A target domain network and teacher network initialization module 200 is used to initialize the target domain network and the teacher network using the target domain low-resolution image and the original domain network;

[0073] The teacher network has the same structure as the target domain network, and the teacher network has been pre-initialized;

[0074] The target domain network learning rate adjustment second module 300 is used to adjust the learning rate of each network layer in the target domain network using the target domain low-resolution image and the teacher network;

[0075] A target domain network adaptive training module 400 is used to perform domain adaptive training on the target domain network using the teacher network;

[0076] The target domain super-resolution image acquisition module 500 is used to acquire a reconstructed target domain super-resolution image using a trained target domain network.

[0077] Beneficial effect: Compared with the prior art, the domain-adaptive image super-resolution reconstruction method and system without original domain single sample disclosed in the present invention realizes domain-adaptive training of the super-resolution network without accessing the original domain samples and only having a single sample in the target domain, thereby reducing the impact of the domain gap on the network performance and effectively improving the performance of the super-resolution network in the target domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0079] Figure 1 It is a schematic diagram of the structure of the original domain network;

[0080] Figure 2 is a schematic diagram of the target domain network structure;

[0081] Figure 3 This is the flowchart of the semantic perception module;

[0082] Figure 4 It is a flow chart of the original domain-free single-image adaptive super-resolution reconstruction method disclosed in the present invention;

[0083] Figure 5 Flowchart for adjusting the learning rate of each network layer in the target domain network using the target domain low-resolution image and the original network;

[0084] Figure 6 Flowchart of random cropping and prior information acquisition operations;

[0085] Figure 7 Flowchart for initializing the target domain network and the teacher network using the target domain low-resolution image and the original domain network;

[0086] Figure 8 Flowchart for random fuzzy data enhancement processing;

[0087] Fig. 9 Flowchart for domain adaptive training of target domain network using teacher network;

[0088] Fig.10 The present invention is a schematic diagram of the composition of the original domain-free single-image adaptive super-resolution reconstruction system disclosed in the present invention. DETAILED DESCRIPTION

[0089] The embodiment of the present invention discloses a method for adaptive image super-resolution reconstruction without a single original domain image. The method uses a single low-resolution image to be reconstructed, that is, a target domain low-resolution image, to train a target domain network to achieve reconstruction of the target domain low-resolution image.

[0090] Before executing the steps of the super-resolution reconstruction method disclosed in the present invention, the target domain network has completed pre-initialization, that is, some of its parameters have been initialized. The pre-initialization of the target domain network is achieved using the pre-trained original domain network. After the original domain network is pre-trained, its parameters are frozen and no longer change in subsequent steps. The original domain network pre-training uses an original domain dataset. The original domain dataset includes multiple high-resolution images and low-resolution images corresponding to the high-resolution images. The original domain dataset is only used for the pre-training of the original domain network and does not participate in the execution of the steps of the method disclosed in the present invention, thereby limiting the scope of use of the images in the original domain dataset.

[0091] The structure of the original domain network is a MambaIR network, including a first convolutional layer, a first deep feature extraction network, and a first upsampling layer; the first convolutional layer is used to extract shallow image features of the original domain network input image, the first deep feature extraction network is used to extract deep image features from shallow image features, and the first deep feature extraction network is composed of multiple RSSGs (Residue State-Space Groups) cascaded in sequence; the first upsampling layer is used to generate a reconstructed super-resolution image based on shallow image features and deep image features. For details of the MambaIR network, please refer to the literature: Guo H, Li J, Dai T, et al. MambaIR: A Simple Baseline for Image Restoration with State-Space Model [C] / / European Conference on Computer Vision. Springer, Cham, 2025. DOI: 10.1007 / 978-3-031-72649-1_13. In this embodiment, the first deep feature extraction network is composed of 6 RSSGs cascaded in sequence, and the structure is as follows Figure 1 shown.

[0092] As Figure 2 As shown, the target domain network includes a second convolutional layer, a semantic perception module, a second deep feature extraction network, and a second upsampling layer; the second convolutional layer is used to extract shallow image features of the input spliced ​​low-resolution image, and the semantic perception module is used to generate semantically guided image features according to the input main semantic information and shallow image features; the second deep feature extraction network is used to extract deep image features in the semantically guided image features, and the second upsampling layer is used to generate a reconstructed super-resolution image according to the shallow image features and the deep image features;

[0093] The first convolutional layer has the same structure as the second convolutional layer; the first deep feature extraction network has the same structure as the second deep feature extraction network; the first upsampling layer has the same structure as the second upsampling layer; that is, compared with the original domain network, the target domain network adds a semantic perception module.

[0094] like Figure 3 As shown, the semantic perception module expands the incoming shallow spliced ​​image features in four directions: horizontal, vertical, reverse horizontal and reverse vertical, to generate four expanded image features; Figure 3 In the figure, the expansion method is described using an image with a pixel size of 2×2. The pixels in the 2×2 image are numbered 1, 2, 3, and 4 in the upper left, upper right, lower left, and lower right order. The expanded image features are ordered 1, 2, 3, and 4 by horizontal expansion, and the expanded image features are ordered 1, 3, 2, and 4 by vertical expansion. The expanded image features are ordered 4, 3, 2, and 1 by reverse horizontal expansion, and the expanded image features are ordered 4, 2, 3, and 1 by reverse vertical expansion. In this embodiment, the low-resolution image size input to the semantic perception module is 48×48. Figure 3 The pixels are numbered in a similar manner, with the upper left pixel numbered 1, the pixels to the right being numbered 2, 3, ..., 48, and the pixels downward being numbered 49, 97, 145, ..., 2257; the lower right pixel is numbered 48*48=2304; the horizontal expansion yields the expanded image features in the order of 1, 2, 3, ..., 2304, and the vertical expansion yields the expanded image features in the order of 1, 49, 97, 145, ..., 2304; the reverse horizontal expansion yields the reverse order of the horizontal expansion, and the reverse vertical expansion yields the reverse order of the vertical expansion. The incoming main semantic information is copied in the channel dimension and the quantity dimension, and aligned with the four expanded image features to generate the copied semantic information; the copied semantic information is spliced ​​before the four expanded image features to generate the image features to be semantically guided; the image features to be semantically guided are input into the State Space Models (SSM) block to generate the scanned image features; the content of the SSM block is detailed in the literature: GuA, Dao T. Mamba: Linear-time sequence modeling with selective state spaces [J]. arXiv preprint arXiv: 2312.00752, 2023. The scanned image features are split, the features corresponding to the main semantic information are cropped, and the guided image features are generated; the guided image features are added in the quantity dimension to generate the merged image features; the merged image features are subjected to shape restoration processing to obtain the semantically guided image features, and the size of the semantically guided image features is consistent with the size of the shallow image features passed into the semantic perception module.

[0095] The pre-training steps of the original domain network are:

[0096] A low-resolution image in the original domain dataset is input into the original domain network, and an original domain super-resolution image is output; a first loss function value is calculated according to the original domain super-resolution image and the high-resolution image in the original domain dataset, and the original domain network is trained by minimizing the first loss function value to obtain a pre-trained original domain network.

[0097] In the embodiment of the present invention, the first loss function value adopts the L1 loss value of the original domain super-resolution image and the high-resolution image in the original domain dataset.

[0098] The pre-initialization steps of the target domain network include:

[0099] The parameters of the second convolutional layer, the second deep feature extraction network, and the second upsampling layer of the target domain network are respectively set to the parameters of the first convolutional layer, the first deep feature extraction network, and the first upsampling layer of the pre-trained original domain network;

[0100] Randomly initialize the parameters of the semantic perception module in the target domain network.

[0101] At this point, the parameter initialization of all parts of the target domain network except the semantic perception module has been completed.

[0102] like Figure 4 As shown, the original domain-free single-image adaptive super-resolution reconstruction method disclosed in the present invention comprises the steps of:

[0103] S1, using the target domain low-resolution image and the original network to adjust the learning rate of each network layer in the target domain network; Figure 5 As shown, the steps include:

[0104] S11. Initialize the number of cropped samples N p , the number of domain specialization layers N s , learning rate increase multiple P i , learning rate reduction ratio P d ;

[0105] In this embodiment, N p Initialized to 30, N s Set to 50% of the number of all network layers in the target domain network, P i Set to 100, P d Set to 0.5;

[0106] S12: Perform N on the low-resolution image in the target domain p Random cropping and prior information acquisition operations are performed to generate N p cropped low-resolution images and corresponding semantic information; pThe cropped low-resolution image is copied 6 times and geometrically enhanced to generate 6*N p geometrically enhanced low-resolution images; 6*N p A geometrically enhanced low-resolution image is input into the original domain network to generate 6*N p super-resolution pseudo-label images; 6*N p The super-resolution pseudo-label images are processed for position restoration respectively to generate 6*N p Position-aligned super-resolution pseudo-label images; 6*N p The super-resolution pseudo-label images aligned at each position are averaged to generate N p pseudo-label images; p The cropped low-resolution images and the corresponding semantic information are input into the target domain network for super-resolution reasoning to generate N p Super-resolution images;

[0107] Calculate N p super-resolution images and N p The L1 loss between the pseudo-label images is calculated, and the gradient is returned to save the gradient of each network layer in the target domain network; the average gradient of each network layer in the target domain network is calculated to obtain the average gradient of each network layer in the target domain network;

[0108] Traverse each network layer of the target domain network. If the average gradient of the current network layer is in the top N of the average gradient size of all network layers of the target domain network, s Among them, the learning rate of the parameters of the current network layer increases by P i times; otherwise, the learning rate of the parameters of the current network layer is reduced by P d .

[0109] In this embodiment, the learning rate of each network layer in the target domain network is set to 1e-5 after pre-initialization. After steps S11-S12, N s The learning rate of the remaining 50% of the network layers is adjusted to 1e-5 (1+100), and the learning rate of the remaining 50% of the network layers is adjusted to 1e-5 (1-0.5). This makes the network layer parameters with large gradients in the target domain network have a larger modification range during the optimization process, while the network layer parameters with small gradients have a smaller modification range during the optimization process, thereby better adapting to the feature distribution of low-resolution images in the target domain.

[0110] like Figure 6 As shown, the random cropping and prior information acquisition operations are:

[0111] The image to be processed (i.e., the target domain low-resolution image in step S12) is randomly cropped into a cropped image with a pixel width of w and a pixel height of h, and a corresponding position mask is generated at the same time; the position mask is consistent with the size of the image to be processed and is used to indicate the position of the cropped image in the image to be processed;

[0112] In this embodiment, the values ​​of w and h are both 48, that is, the pixel size of the cropped image is i48, 48). The size of the image to be processed is (W, H), the size of the position mask is also (W, H), and the pixel value of the position corresponding to the cropped image on the position mask is set to 1, and the pixel values ​​of the other positions are set to 0.

[0113] The position mask and the image to be processed are input into the Alpha-CLIP network to generate semantic information corresponding to the cropped image.

[0114] For details of the Alpha-CLIP network, see the literature: Sun z, Fang Y, Wu T, et al. Alpha-CLIP: A CLIP Model Focusing on Wherever You Want [J]. IEEE, 2023. DOI: 10.1109 / CVPR52733.2024.01237.).

[0115] Geometric enhancement is to process images by using geometric transformation. The geometric transformations used in the embodiment of the present invention include horizontal flipping, vertical flipping, clockwise rotation of 90 degrees, clockwise rotation of 180 degrees, and clockwise rotation of 270 degrees. In step S12, the six copied cropped low-resolution images are subjected to geometric enhancement processing as follows: the first copied image remains unchanged, and the other five copied images are subjected to the above five different geometric transformations respectively, to obtain six geometrically enhanced images.

[0116] The corresponding position restoration process is the inverse transformation of the geometric transformation, that is, if horizontal flipping is used in the geometric enhancement process, horizontal flipping is performed again in the position restoration process; if 90 degrees clockwise rotation is used in the geometric enhancement process, counterclockwise rotation is performed 90 degrees in the position restoration process. Similarly, the inverse transformation of vertical flipping is another vertical flipping, and the inverse transformation of clockwise rotation is to rotate counterclockwise by the same angle.

[0117] S2. Initialize the target domain network and the teacher network using the target domain low-resolution image and the original domain network; the teacher network has the same structure as the target domain network, and the teacher network has been pre-initialized;

[0118] The pre-initialization of the teacher network is as follows: the parameters of the second convolutional layer, the second deep feature extraction network, and the second upsampling layer of the teacher network are respectively set to the parameters of the first convolutional layer, the first deep feature extraction network, and the first upsampling layer of the pre-trained original domain network;

[0119] Randomly initialize the parameters of the semantic perception module in the teacher network.

[0120] like Figure 7 As shown, the target domain low-resolution image and the original domain network are used to initialize the target domain network and the teacher network, including the steps of:

[0121] S21, generating a main low-resolution image LR based on the target domain low-resolution image p , secondary low-resolution image LR a and the main semantic information S p , specifically including steps S211-S213:

[0122] S211, perform two random cropping and prior information acquisition operations on the target domain low-resolution image to obtain two cropped low-resolution images and And the corresponding semantic information S a and S b ;

[0123] S212, for cropping low-resolution images and Perform random fuzzy data enhancement processing to generate a fuzzy enhanced cropped low-resolution image and

[0124] S213, Random Assignment and One of them is used as the main low-resolution image LR p , the other is a low-resolution image LR a ; At the same time, the semantic information corresponding to the main low-resolution image is designated as the main semantic information S p ;

[0125] p=Random_select_index(a,b)s.tp∈{a,b}

[0126] Random_select_index(a, b) means selecting the sequence number p of the main low-resolution image from the random index set {a, b};

[0127] In this embodiment, the low-resolution image is cropped and The pixel size of the main low-resolution image LR is (48, 48).p and a low-resolution image LR a The size is (2,3,48,48), where 2 is the batch size, 3 is the number of channels, and the last two digits are the height and width of the input feature. p The size is (2, 512).

[0128] like Figure 8 As shown, the random fuzzy data enhancement process is:

[0129] The image P to be processed is downsampled and upsampled twice to generate a forged image P fake ;

[0130] Generate a random mask Mask, the size of which is consistent with the size of the image P to be processed, and the random mask Mask is divided into a grid with a pixel width of w1 and a pixel height of h1, and the pixel value in each grid is set to 1 with a probability of 40%; in the embodiment of the present application, w1=12 and h1=12 are set;

[0131] The image BAug(P) after random fuzzy data enhancement is:

[0132] BAug(P)=Mask*P+Rev(Mask)*P fake

[0133] Rev(Mask) means to perform a zero-one flip operation on the Mask, that is, to swap each pixel value in the random mask, the pixel value of 1 is swapped to 0, and the pixel value of 0 is swapped to 1.

[0134] In step S212, the low-resolution image is cropped and Perform random fuzzy data enhancement processing respectively, namely:

[0135]

[0136] S22, using the target domain network to LR the main low-resolution image p , secondary low-resolution image LR a and the main semantic information S p Processing is performed to generate the main super-resolution image SR p ; Use the original domain network to LR the main low-resolution image p Process and generate pseudo label PL S ;

[0137] Generating a primary super-resolution image SR p The specific steps include:

[0138] S2211, the main low-resolution image LRp and a low-resolution image LR a Stitching is performed in a random order to generate a stitched low-resolution image CLR;

[0139] CLR=Random_cat(LR p , L.R. a )

[0140] Random_cat means random sequence splicing operation;

[0141] In this embodiment, the size of the stitched low-resolution image CLR is (2, 3, 48, 48×2).

[0142] S2212, input the spliced ​​low-resolution image CLR into the second convolutional layer in the target domain network, and input the main semantic information S p The semantic perception module in the target domain network is input, and the second upsampling layer in the target domain network outputs the spliced ​​super-resolution image CSR;

[0143] CSR=M T (CLR, S p )

[0144] M T (CLR, S p ) represents the target domain network for the concatenated low-resolution image CLR and the main semantic information S p The result is the spliced ​​super-resolution image CSR;

[0145] S2213, split the spliced ​​super-resolution image CSR, retaining the main semantic information S p The corresponding super-resolution pixel area is used as the main super-resolution image SR p ;

[0146] SR p =Split_and_get_primary(CSR)

[0147] Split_and_get_primary represents the split operation, which splits the primary super-resolution image SR from the spliced ​​super-resolution image CSR p In this embodiment, the size of the super-resolution image CSR is (2, 3, 48 × Scale up , 48×2×Scale up ), Scale up is the super-resolution multiple. In this embodiment, Scale up Set to 4. Main super-resolution image SR p The size is (2,3,48×Scaleup , 48×Scale up ).

[0148] Since the target domain has only a single sample, namely the low-resolution image of the target domain, the distribution diversity of the target domain training samples is limited, which may cause the network to experience mode collapse and affect the domain adaptive training efficiency of the target domain network. The present invention divides the cropped low-resolution image into a main low-resolution image and a secondary low-resolution image, and regards the secondary low-resolution image as a background feature. By combining the main low-resolution image with the background feature (secondary low-resolution image), the diversity of the distribution of the target domain training samples can be expanded.

[0149] The pseudo label PL is generated S Specifically including steps S2221-S2223:

[0150] S2221, the main low-resolution image LR p Duplicate 6 copies and perform geometric enhancement to generate 6 geometrically enhanced low-resolution images

[0151]

[0152] GAug means geometric enhancement processing;

[0153] Input 6 geometrically enhanced low-resolution images into the original domain network to generate 6 super-resolution images

[0154]

[0155] in Representing the original domain network for geometrically enhanced low-resolution images Operations;

[0156] S2222, performing position restoration processing on the six super-resolution images respectively to generate six position-aligned super-resolution images;

[0157] S2223, calculate the average value of the super-resolution images aligned at the 6 positions and generate a pseudo label PL S ;

[0158]

[0159] Among them, raug means position restoration processing, and mean means calculating the average value;

[0160] The geometry enhancement processing and position restoration processing in steps S2231-S2233 are the same as the geometry enhancement processing and position restoration processing in step S12.

[0161] S23, according to the pseudo label PL S and the main super-resolution image SR p Calculating a second loss function value, and optimizing the target domain network by minimizing the second loss function value;

[0162] In this embodiment, the second loss function value Loss warmup Using pseudo-label PL S and the main super-resolution image SR p The L1 loss value is:

[0163] Loss warmup =L1(SR p , PL S )

[0164] L1 represents the L1 loss value calculation operation.

[0165] S24. Synchronously update the teacher network parameters using exponential averaging:

[0166]

[0167] Represent the teacher network parameters before and after updating, is the parameter of the target domain network, and γ represents the preset retention coefficient.

[0168] The structure of the teacher network is the same as that of the target domain network. The teacher network parameters can be updated by directly copying the target domain network parameters as the teacher network parameters. However, when the target domain network encounters samples that deviate from the center distribution, the gradient generated by this method may deviate from the learning goal. Therefore, the present invention adopts the Exponential Moving Average (EMA) method to synchronously update the teacher network parameters, and uses the information of the past multiple steps to optimize the gradient of the current step and update the parameters, so that the parameter update is more stable.

[0169] Steps S1-S2 use the original domain network to initialize the target domain network. Because the parameters in the original domain network are for the image feature distribution in the original domain data set and are not adjusted in the domain adaptive training, in order to further improve the performance label of the target domain network on the target domain image, the present invention uses the teacher network to guide the target domain network during the domain adaptive training of the target domain network.

[0170] S3. Use the target domain low-resolution image and the teacher network to adjust the learning rate of each network layer in the target domain network; specifically including:

[0171] S31, initialize the number of cropped samples N p , the number of domain specialization layers Ns , learning rate increase multiple P i , learning rate reduction ratio P d ;

[0172] In this embodiment, N p Initialized to 30, N s Set to 60% of the number of all network layers in the target domain network. i Set to 4, P d Set to 0.5;

[0173] S32, perform N on the target domain low-resolution image p Random cropping and prior information acquisition operations are performed to generate N p cropped low-resolution images and corresponding position masks; N p cropped low-resolution images and N p The position masks are copied 6 times and geometrically enhanced to generate 6*N p geometrically enhanced low-resolution images and 6*N p geometric enhancement position masks; copy the target domain low-resolution image 6*N p times, and performs geometric enhancement processing to generate 6*N p The number of aligned target domain low-resolution images; 6*N p The geometrically enhanced position mask together with 6*N p The number of aligned target domain low-resolution images is sent to the Alpha-CLIP network to generate 6*N p semantic information; 6*N p geometrically enhanced low-resolution images together with 6*N p The semantic information is input into the teacher network together to generate 6*N p super-resolution pseudo-label images; 6*N p Super-resolution pseudo-label images are processed for position restoration to generate 6*N p Position-aligned super-resolution pseudo-label images; 6*N p The super-resolution pseudo-label images aligned at each position are averaged to generate N p pseudo-label images; p The cropped low-resolution images and their corresponding semantic information are input into the target domain network for super-resolution reasoning to generate N p Super-resolution images;

[0174] Calculate N p super-resolution images and N pThe L1 loss between the pseudo-label images is calculated, and the gradient is returned to save the gradient of each network layer in the target domain network; the average gradient of each network layer in the target domain network is calculated to obtain the average gradient of each network layer in the target domain network;

[0175] Traverse each network layer of the target domain network. If the average gradient of the current network layer is in the top N of the average gradient size of all network layers of the target domain network, s Among them, the learning rate of the parameters of the current network layer increases by P i times, otherwise, the learning rate of the parameters of the current network layer is reduced by P d .

[0176] In step S1, the learning rate of the target domain network is adjusted using the original domain network, and in step S3, the learning rate of the target domain network is adjusted again using the teacher network.

[0177] S4, using the teacher network to perform domain adaptive training on the target domain network; e.g. Fig. 9 As shown, the steps include:

[0178] S41, generating a main low-resolution image LR according to the target domain low-resolution image p , secondary low-resolution image LR a and the main semantic information S p ; The specific steps are the same as step S21, but due to the multiple random operations in the specific process, the results of S41 and S21 are not exactly the same;

[0179] LR of the main low-resolution image using the target domain network p , secondary low-resolution image LR a and the main semantic information S p Processing is performed to generate the main super-resolution image SR p ;

[0180] S42, use the teacher network to process the low-resolution image of the target domain and generate a pseudo label PL Te , specifically including:

[0181] The target domain low-resolution image is cropped, and the position and size of the cropping are consistent with the main low-resolution image LR p The cropped low-resolution image and the corresponding position mask are generated; the cropped low-resolution image, the corresponding position mask and the target domain low-resolution image are copied 6 times respectively, and geometric enhancement is performed to generate 6 geometrically enhanced cropped target domain low-resolution images 6 geometry-enhanced position masks and 6 geometrically enhanced target domain low-resolution images The 6 geometrically enhanced position masks and 6 geometrically enhanced target domain low-resolution images are input into the Alpha-CLIP network to generate 6 semantic prior information S Te ; 6 geometrically enhanced cropped target domain low-resolution images and 6 semantic prior information S Te Send it to the teacher network to generate 6 super-resolution images The six super-resolution images are processed for position restoration to generate a super-resolution image with position alignment; the average value of the super-resolution images with position alignment is calculated to generate a pseudo label PL Te ;

[0182] Step S42 can be expressed by the following calculation formula:

[0183]

[0184] in Represents the teacher network for 6 geometrically enhanced cropped target domain low-resolution images and 6 semantic prior information S Te , and 6 super-resolution images are obtained

[0185] S43, according to the pseudo label PL Te and the main super-resolution image SR p A third loss function value is calculated, and the target domain network is optimized by minimizing the third loss function value.

[0186] In this embodiment, the third loss function value Loss formal Using pseudo-label PL Te and the main super-resolution image SR p The L1 loss value is:

[0187] Loss formal =L1(SR p , PL Te )

[0188] S5. Use the trained target domain network to obtain the reconstructed target domain super-resolution image. The specific steps are as follows:

[0189] In order to save video memory, the target domain low-resolution image is split into multiple image blocks, and corresponding position masks are generated; in this embodiment, the size of each split image block is 200×200; the target domain low-resolution image and the position mask are input into the Alpha-CLIP network to generate corresponding semantic prior information; each split image block together with the semantic prior information is input into the trained target domain network to generate multiple split target domain super-resolution images; the multiple split target domain super-resolution images are spliced ​​in the spatial dimension to generate a reconstructed target domain super-resolution image.

[0190] This embodiment uses the Olympus data branch in the DRealSR dataset as the original domain dataset, and the Panasonic data branch in the DRealSR dataset as the target domain low-resolution image, and tests and verifies the original domain-free single-image adaptive image super-resolution reconstruction method disclosed in the present invention. The experimental results show that after applying the method disclosed in the present invention, the peak signal-to-noise ratio of the target domain network performance indicator is improved by 0.69 (db).

[0191] Introduction to the DRealSR dataset Reference: Wei P, Xie Z, Lu H, et al. Component Divide-and-Conquer for Real-World Image Super-Resolution[C] / / 2020.DOI:10.1007 / 978-3-030-58598-3_7.

[0192] The embodiment of the present invention also discloses a system for implementing the above-mentioned method for adaptive image super-resolution reconstruction without original domain single-image. Fig.10 As shown, including:

[0193] The target domain network learning rate adjustment first module 100 is used to adjust the learning rate of each network layer in the target domain network using the target domain low-resolution image and the original network according to step S1;

[0194] A target domain network and teacher network initialization module 200 is used to initialize the target domain network and the teacher network using the target domain low-resolution image and the original domain network according to step S2;

[0195] The teacher network has the same structure as the target domain network, and the teacher network has been pre-initialized;

[0196] The target domain network learning rate adjustment second module 300 is used to adjust the learning rate of each network layer in the target domain network using the target domain low-resolution image and the teacher network according to step S3;

[0197] A target domain network adaptive training module 400 is used to perform domain adaptive training on the target domain network using the teacher network according to step S4;

[0198] The target domain super-resolution image acquisition module 500 is used to acquire the reconstructed target domain super-resolution image using the trained target domain network according to step S5.

[0199] The present invention provides a method and idea for adaptive super-resolution reconstruction of images without an original domain. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A method for self-adaptive image super-resolution reconstruction without original domain single-sample domain, characterized in that: Includes steps: S1, using the target domain low-resolution image and the original network to adjust the learning rate of each network layer in the target domain network; The original domain network has completed pre-training; the target domain network has completed pre-initialization; The structure of the original domain network is a MambaIR network, including a first convolutional layer, a first deep feature extraction network, and a first upsampling layer; the first convolutional layer is used to extract shallow image features of the original domain network input image, the first deep feature extraction network is used to extract deep image features from shallow image features, and the first deep feature extraction network is composed of a plurality of RSSGs cascaded in sequence; the first upsampling layer is used to generate a reconstructed super-resolution image according to shallow image features and deep image features; The target domain network includes a second convolutional layer, a semantic perception module, a second deep feature extraction network, and a second upsampling layer; the second convolutional layer is used to extract shallow image features of the input spliced ​​low-resolution image, and the semantic perception module is used to generate semantically guided image features according to the input main semantic information and shallow image features; the second deep feature extraction network is used to extract deep image features in the semantically guided image features, and the second upsampling layer is used to generate a reconstructed super-resolution image according to the shallow image features and the deep image features; The first convolutional layer has the same structure as the second convolutional layer; the first deep feature extraction network has the same structure as the second deep feature extraction network; the first upsampling layer has the same structure as the second upsampling layer; The semantic perception module expands the incoming shallow image features in four directions: horizontal, vertical, reverse horizontal and reverse vertical, to generate four expanded image features; at the same time, the incoming main semantic information is copied in the channel dimension and the quantity dimension, aligned with the four expanded image features, to generate copied semantic information; the copied semantic information is spliced ​​before the four expanded image features to generate image features to be semantically guided; the image features to be semantically guided are input into the SSM module to generate scanned image features; the scanned image features are split, the features at the corresponding positions of the main semantic information are cropped, and guided image features are generated; the guided image features are added in the quantity dimension to generate merged image features; the merged image features are subjected to shape restoration processing to obtain semantically guided image features, and the size of the semantically guided image features is consistent with the size of the shallow image features passed to the semantic perception module; S2. Initialize the target domain network and the teacher network using the target domain low-resolution image and the original domain network; the teacher network has the same structure as the target domain network, and the teacher network has been pre-initialized; S3, using the target domain low-resolution image and the teacher network to adjust the learning rate of each network layer in the target domain network; S4, use the teacher network to perform domain adaptive training on the target domain network; S5. Using the trained target domain network, a reconstructed target domain super-resolution image is obtained. The steps are as follows: The target domain low-resolution image is split into multiple image blocks and the corresponding position masks are generated; the target domain low-resolution image and the position mask are input into the Alpha-CLIP network to generate the corresponding semantic prior information; Each split image block together with the semantic prior information is input into the trained target domain network to generate multiple split target domain super-resolution images; the multiple split target domain super-resolution images are spliced ​​in the spatial dimension to generate a reconstructed target domain super-resolution image.

2. The image super-resolution reconstruction method according to claim 1, characterized in that: Pre-training of the original domain network includes: Inputting a low-resolution image in the original domain data set into the original domain network, and outputting an original domain super-resolution image; calculating a first loss function value according to the original domain super-resolution image and the high-resolution image in the original domain data set, and training the original domain network by minimizing the first loss function value to obtain a pre-trained original domain network; The original domain dataset includes a plurality of high-resolution images and low-resolution images corresponding to the high-resolution images.

3. The image super-resolution reconstruction method according to claim 1, characterized in that: The pre-initialization of the target domain network and the teacher network includes: The parameters of the second convolutional layer, the second deep feature extraction network, and the second upsampling layer of the target domain network and the teacher network are respectively set to the parameters of the first convolutional layer, the first deep feature extraction network, and the first upsampling layer of the pre-trained original domain network; The parameters of the semantic perception modules in the target domain network and the teacher network are randomly initialized.

4. The image super-resolution reconstruction method according to claim 1, characterized in that: The method of adjusting the learning rate of each network layer in the target domain network by using the target domain low-resolution image and the original network includes: S11. Initialize the number of cropped samples N p , the number of domain specialization layers N s , learning rate increase multiple P i , learning rate reduction ratio P d ; S12: Perform N on the low-resolution image in the target domain p Random cropping and prior information acquisition operations are performed to generate N p cropped low-resolution images and corresponding semantic information; p The cropped low-resolution image is copied 6 times and geometrically enhanced to generate 6*N p geometrically enhanced low-resolution images; 6*N p A geometrically enhanced low-resolution image is input into the original domain network to generate 6*N p super-resolution pseudo-label images; 6*N p The super-resolution pseudo-label images are processed for position restoration respectively to generate 6*N p Position-aligned super-resolution pseudo-label images; 6*N p The super-resolution pseudo-label images aligned at each position are averaged to generate N p pseudo-label images; p The cropped low-resolution images and the corresponding semantic information are input into the target domain network for super-resolution reasoning to generate N p Super-resolution images; Calculate N p super-resolution images and N p The L1 loss between the pseudo-label images is calculated, and the gradient is returned to save the gradient of each network layer in the target domain network; the average gradient of each network layer in the target domain network is calculated to obtain the average gradient of each network layer in the target domain network; Traverse each network layer of the target domain network. If the average gradient of the current network layer is in the top N of the average gradient size of all network layers of the target domain network, s Among them, the learning rate of the parameters of the current network layer increases by P i times, otherwise, the learning rate of the parameters of the current network layer is reduced by P d .

5. The image super-resolution reconstruction method according to claim 4, characterized in that: The random cropping and prior information acquisition operations are: The image to be processed is randomly cropped into a cropped image with a pixel width of w and a pixel height of h, and a corresponding position mask is generated at the same time; the position mask is consistent with the size of the image to be processed and is used to indicate the position of the cropped image in the image to be processed; The position mask and the image to be processed are input into the Alpha-CLIP network to generate semantic information corresponding to the cropped image.

6. The image super-resolution reconstruction method according to claim 1, characterized in that: The method of initializing the target domain network and the teacher network using the target domain low-resolution image and the original domain network includes the following steps: S21, generating a main low-resolution image LR based on the target domain low-resolution image p , secondary low-resolution image LR a and the main semantic information S p , specifically including the steps: S211, perform two random cropping and prior information acquisition operations on the target domain low-resolution image to obtain two cropped low-resolution images and And the corresponding semantic information S a and S b ; S212, for cropping low-resolution images and Perform random fuzzy data enhancement processing to generate a fuzzy enhanced cropped low-resolution image and S213, Random Assignment and One of them is used as the main low-resolution image LR p , the other is a low-resolution image LR a ; At the same time, the semantic information corresponding to the main low-resolution image is designated as the main semantic information S p ; S22, using the target domain network to LR the main low-resolution image p , secondary low-resolution image LR a and the main semantic information S p Processing is performed to generate the main super-resolution image SR p ; Use the original domain network to LR the main low-resolution image p Process and generate pseudo label PL S ; Generating a primary super-resolution image SR p The specific steps include: S2211, the main low-resolution image LR p The secondary low-resolution image is spliced ​​in a random order to generate a spliced ​​low-resolution image CLR; S2212, input the spliced ​​low-resolution image CLR into the second convolutional layer in the target domain network, and input the main semantic information S p The semantic perception module in the target domain network is input, and the second upsampling layer in the target domain network outputs the spliced ​​super-resolution image CSR; S2213, split the spliced ​​super-resolution image CSR, retaining the main semantic information S p The corresponding super-resolution pixel area is used as the main super-resolution image SR p ; The pseudo label PL is generated S Specifically include: S2221, the main low-resolution image LR p Duplicate 6 copies and perform geometric enhancement to generate 6 geometrically enhanced low-resolution images Input 6 geometrically enhanced low-resolution images into the original domain network to generate 6 super-resolution images S2222, performing position restoration processing on the six super-resolution images respectively to generate six position-aligned super-resolution images; S2223, calculate the average value of the super-resolution images aligned at the 6 positions and generate a pseudo label PL S ; S23, according to the pseudo label PL S and the main super-resolution image SR p Calculating a second loss function value, and optimizing the target domain network by minimizing the second loss function value; S24. Synchronously update the teacher network parameters using exponential averaging: Represent the teacher network parameters before and after updating, is the parameter of the target domain network, and γ represents the preset retention coefficient.

7. The image super-resolution reconstruction method according to claim 6, characterized in that: The random fuzzy data enhancement process is as follows: The image P to be processed is downsampled and upsampled twice to generate a forged image P fake ; Generate a random mask Mask. The size of the random mask Mask is consistent with the size of the image P to be processed. The random mask Mask is divided into a grid with a pixel width of w1 and a pixel height of h1. The pixel value in each grid is set to 1 with a probability of 40%; The image BAug(P) after random fuzzy data enhancement is: BAug(P)=Mask*P+Rev(Mask)*P fake Rev(Mask) means to perform a zero-one flip operation on the Mask, that is, to swap each pixel value in the random mask, the pixel value of 1 is swapped to 0, and the pixel value of 0 is swapped to 1.

8. The image super-resolution reconstruction method according to claim 1, characterized in that: The method of adjusting the learning rate of each network layer in the target domain network by using the target domain low-resolution image and the teacher network includes: S31, initialize the number of cropped samples N p , the number of domain specialization layers N s , learning rate increase multiple P i , learning rate reduction ratio P d ; S32, perform N on the target domain low-resolution image p Random cropping and prior information acquisition operations are performed to generate N p cropped low-resolution images and corresponding position masks; p cropped low-resolution images and N p The position masks are copied 6 times and geometrically enhanced to generate 6*N p geometrically enhanced low-resolution images and 6*N p geometric enhancement position masks; copy the target domain low-resolution image 6*N p times, and performs geometric enhancement processing to generate 6*N p The number of aligned target domain low-resolution images; 6*N p The geometrically enhanced position mask together with 6*N p The number of aligned target domain low-resolution images is sent to the Alpha-CLIP network to generate 6*N p semantic information; 6*N p geometrically enhanced low-resolution images together with 6*N p The semantic information is input into the teacher network together to generate 6*N p super-resolution pseudo-label images; 6*N p Super-resolution pseudo-label images are processed for position restoration to generate 6*N p Position-aligned super-resolution pseudo-label images; 6*N p The super-resolution pseudo-label images aligned at each position are averaged to generate N p pseudo-label images; p The cropped low-resolution images and their corresponding semantic information are input into the target domain network for super-resolution reasoning to generate N p Super-resolution images; Calculate N p super-resolution images and N p The L1 loss between the pseudo-label images is calculated, and the gradient is returned to save the gradient of each network layer in the target domain network; the average gradient of each network layer in the target domain network is calculated to obtain the average gradient of each network layer in the target domain network; Traverse each network layer of the target domain network. If the average gradient of the current network layer is in the top N of the average gradient size of all network layers of the target domain network, s Among them, the learning rate of the parameters of the current network layer increases by P i times, otherwise, the learning rate of the parameters of the current network layer is reduced by P d .

9. The image super-resolution reconstruction method according to claim 1, characterized in that: The method of using the teacher network to perform domain adaptive training on the target domain network includes the following steps: S41, generating a main low-resolution image LR according to the target domain low-resolution image p , secondary low-resolution image LR a and the main semantic information S p ; LR of the main low-resolution image using the target domain network p , secondary low-resolution image LR a and the main semantic information S p Processing is performed to generate the main super-resolution image SR p ; S42, use the teacher network to process the low-resolution image of the target domain and generate a pseudo label PL Te , specifically including: The target domain low-resolution image is cropped, and the cropped position and size are consistent with the main low-resolution image to generate a cropped low-resolution image and a corresponding position mask; the cropped low-resolution image, the corresponding position mask and the target domain low-resolution image are copied 6 times respectively, and geometrically enhanced to generate 6 geometrically enhanced cropped target domain low-resolution images 6 geometry-enhanced position masks and 6 geometrically enhanced target domain low-resolution images The 6 geometrically enhanced position masks and 6 geometrically enhanced target domain low-resolution images are input into the Alpha-CLIP network to generate 6 semantic prior information S Te ; 6 geometrically enhanced cropped target domain low-resolution images and 6 semantic prior information S Te Send it to the teacher network to generate 6 super-resolution images The six super-resolution images are processed for position restoration to generate a super-resolution image with position alignment; the average value of the super-resolution images with position alignment is calculated to generate a pseudo label PL Te ; S43, according to the pseudo label PL Te and the main super-resolution image SR p Calculating a third loss function value, and optimizing the target domain network by minimizing the third loss function value; S44. Synchronously update the teacher network parameters using exponential averaging: Represent the teacher network parameters before and after updating, is the parameter of the target domain network, and γ represents the preset retention coefficient.

10. A non-original domain single-sample domain adaptive image super-resolution reconstruction system, characterized in that: include: A target domain network learning rate adjustment first module (100) is used to adjust the learning rate of each network layer in the target domain network using the target domain low-resolution image and the original network; A target domain network and teacher network initialization module (200) is used to initialize the target domain network and the teacher network using the target domain low-resolution image and the original domain network; The teacher network has the same structure as the target domain network, and the teacher network has been pre-initialized; A target domain network learning rate adjustment second module (300) is used to adjust the learning rate of each network layer in the target domain network using the target domain low-resolution image and the teacher network; A target domain network adaptive training module (400) is used to perform domain adaptive training on the target domain network using the teacher network; The target domain super-resolution image acquisition module (500) is used to acquire a reconstructed target domain super-resolution image using a trained target domain network.

Images