Self-supervised image super-resolution method based on active displacement system

Abstract

The invention discloses a self-supervised image super-resolution method based on an active displacement imaging system. The method comprises the following steps: 1) constructing the active displacement imaging system; 2) controlling the image sensor to carry out multi-frame sub-pixel-level image acquisition in the X / Y direction by taking 1 / S pixel as a step length, and recording the displacement coordinate of each frame; 3) calculating inter-frame relative displacement based on the displacement information, and taking the inter-frame relative displacement as a reference for image alignment and reconstruction; 4) extracting features of each frame of image by using an encoder, and mapping low-resolution features to a high-resolution grid by using a displacement encoding pixel mapping module; 5) performing pixel aggregation on the high-resolution features of all frames to form a unified high-resolution feature map; according to the method, the image detail reduction capability and the spatial resolution can be effectively improved, and the reconstruction precision and the practicability of an imaging system in a complex application scene can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image super-resolution reconstruction method that combines sub-pixel displacement control with self-supervised learning. Specifically, the method is a self-supervised image super-resolution method based on an active displacement imaging system, which can be used in application scenarios requiring high-resolution image reconstruction, such as industrial detection, computational imaging, remote sensing, and medical imaging. Background Art

[0002] Image super-resolution technology aims to recover higher spatial resolution image details from low-resolution images and is an important research direction in the field of image restoration and enhancement. Existing multi-frame super-resolution methods typically rely on the inter-frame displacements present in natural image sequences to supplement pixel information and improve image quality. However, these methods rely heavily on accurate displacement estimation, and their performance is easily affected by factors such as image noise, compression artifacts, and motion blur. This can lead to the accumulation of displacement estimation errors, which in turn affects reconstruction accuracy and robustness.

[0003] In recent years, some research has attempted to combine deep learning methods with multi-frame super-resolution techniques. However, these methods mostly use supervised training and rely on high-quality, high-resolution labeled images, making them difficult to apply to unlabeled data in real-world imaging scenarios. Furthermore, traditional image acquisition systems often cannot achieve controlled displacement with sub-pixel accuracy, limiting the acquisition of spatial redundancy between images and thus restricting the ability to recover image detail.

[0004] Therefore, how to construct a high-precision multi-frame super-resolution method that combines controllable acquisition and self-supervised learning without high-resolution labeled images has become a key issue that needs to be urgently solved in the current field of image processing and imaging system integration. Summary of the Invention

[0005] In order to address the shortcomings of the above-mentioned existing technologies, the present invention proposes a self-supervised image super-resolution method based on an active displacement imaging system, so that in the absence of high-resolution labeled images, by precisely controlling the sub-pixel displacement of the image sensor, image sequences with spatially redundant information can be collected. In addition, by combining deep feature encoding, displacement mapping and self-supervised learning mechanisms, the method can effectively restore image details and achieve super-resolution reconstruction, thereby improving the clarity and structural restoration of the reconstructed image, enhancing the adaptability and versatility of the imaging system in complex scenes, and meeting the high-resolution application requirements of industrial inspection, precision measurement and scientific imaging.

[0006] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0007] The present invention discloses a self-supervised image super-resolution method based on an active displacement imaging system. The active displacement system comprises an image sensor, a host computer processing module, a lens, a controller, and a piezoelectric displacement stage. The image sensor is fixed to the piezoelectric displacement stage, and the centerline of the image sensor and the centerline of the lens are aligned, serving as the initial position of the piezoelectric displacement stage. The self-supervised image super-resolution method is characterized in that the following steps are performed:

[0008] Step 1: Define the number of frames currently collected as t and initialize t=1;

[0009] After the lens focuses on the object to be measured, the image sensor images the object at the initial position and shifts the acquired frame t-1 of the object to be measured. As a reference frame Then send it to the host computer processing module and record the initial position coordinates ; Among them, H represents the height of the shifted image, W represents the width of the shifted image, and C represents the number of channels of the shifted image

[0010] Step 2: The host computer processing module receives the After the frame shift image, the t-th displacement instruction is sent to the controller;

[0011] The controller generates the t-th displacement accuracy parameters in the X direction and the Y direction according to the t-th displacement instruction and transmits them to the piezoelectric displacement stage;

[0012] The piezoelectric displacement platform is configured to move in the X and Y directions according to the received t-th displacement accuracy parameter. The pixel size is a step displacement, which drives the image sensor to move for the tth time. After reaching the preset position, the host computer processing module uses the image sensor to collect the tth frame shift image of the observed object. After that, send it to the host computer processing module and record the t-th frame shift image Location coordinates ; Thus, we get an image sequence of T frames ,in, Indicates the magnification factor, Indicates the total number of shifted images;

[0013] Step 3: Use formula (1) to calculate the image sequence The t-th frame shift image Relative to the reference frame Inter-frame displacement information ;

[0014] (1)

[0015] In formula (1), ( ) represents the relative displacement mapping function;

[0016] Step 4: Use the encoder to Perform feature extraction to obtain the depth feature of the tth frame ; Where N represents the number of channels of the encoder output feature;

[0017] Step 5: Based on the inter-frame displacement information right Perform spatial relocation to obtain high-resolution features of the tth frame , thus obtaining a high-resolution feature map ;

[0018] Step 6: Use the decoder to Reconstruct and obtain super-resolution images ;

[0019] Step 7: Use formula (5) to construct self-supervised L1 loss :

[0020] (6)

[0021] In formula (6), represents the downsampling operation, Obtaining a supervision frame independently collected under the same scene for the active displacement system; represents the image registration operation based on displacement compensation, Supervisory frame Displacement compensation in X and Y directions; represents the L1 norm;

[0022] Step 8: Jointly optimize and update the network parameters of the encoder and decoder through the back propagation algorithm; until Until convergence, the optimal image super-resolution reconstruction model is obtained, which is used to super-resolve the displacement images of the collected observation objects.

[0023] The self-supervised image super-resolution method based on active displacement imaging system of the present invention is also characterized in that the encoder is obtained by using formula (2) :

[0024] (2)

[0025] In formula (2), represents the convolutional layer, represents the K-th cascaded deep channel perception unit, represents another convolutional layer; Represents the parameters of the encoder.

[0026] Furthermore, the step 5 includes:

[0027] Step 5.1: Use formula (4) to Processing is performed to obtain the high-resolution features of the tth frame , thus obtaining the high-resolution features of T frames , :

[0028] (4)

[0029] In formula (4), Represents the displacement coded pixel mapping module;

[0030] Step 5.2, The high-resolution features in the image are aggregated and fused into a unified high-resolution feature map. ;

[0031] Furthermore, the displacement-coded pixel mapping module in step 5.1 includes:

[0032] Step 1: Based on the set magnification S, the reference frame Corresponding image size Construct a dimension High-resolution grid G;

[0033] Step 2: Based on the inter-frame displacement information , use bilinear interpolation to convert the low-resolution feature map of T frame The position of each pixel feature point in Remap to the corresponding pixel feature point target position in the high-resolution grid G , thus obtaining high-resolution features of T frames ;

[0034] 6. The self-supervised image super-resolution method based on active displacement imaging system according to claim 1, characterized in that:

[0035] The decoder is obtained using formula (5) :

[0036] (5)

[0037] In formula (5), Represents the parameters of the decoder; represents a residual connection.

[0038] Furthermore, each cascaded deep channel perception unit includes: several convolutional layers, global pooling layers, the first fully connected layer , the second fully connected layer And Sigmoid activation function module;

[0039] When k=1, after the input feature j is input into the kth cascade deep channel perception unit, it first passes through several consecutive convolutional layers to extract spatial features and output the convolution feature ; Then pass the global pooling layer to The dimension on each channel is The two-dimensional features are compressed to obtain the scalar value of each channel, thus obtaining the dimension The global feature vector is input to the first fully connected layer It is used to compress the channel dimension of the global feature vector to 1 / S times of the original one and output a dimension of Efficient feature representation vector of ;

[0040] The efficient feature representation vector is passed through the second fully connected layer Restore its channel dimension to its original size and output dimension After the channel weight vector is obtained, the channel weight vector is normalized by the sigmoid function to obtain the channel attention weight vector of the kth cascade ; Thus, the deep channel perception output by the k-th cascaded deep channel perception unit is obtained using formula (3) :

[0041] (3)

[0042] When k=2,3,…,K, the deep channel perception output by the k-1th cascaded deep channel perception unit is Input the deep channel perception unit of the kth cascade for processing, and obtain the deep channel perception output by the kth cascade deep channel perception unit , so that the K-th cascaded deep channel perception unit outputs the final deep channel perception .

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. This invention utilizes active sub-pixel displacement acquisition, significantly improving the accuracy and complementarity of inter-frame information. By securing the image sensor to a piezoelectric displacement stage and using a controller to precisely drive the sensor in the X and Y directions with a step size of 1 / S pixels, this method achieves controllable, regular, sub-pixel spatial displacement between image frames. Compared to existing methods based on natural photography or estimated displacement, this technical solution avoids the uncertainty and cumulative error of traditional optical flow or motion estimation, effectively improving the quality of spatially redundant information between images and fundamentally ensuring the accuracy and stability of subsequent super-resolution reconstruction.

[0045] 2. This invention employs a self-supervised learning strategy to eliminate the reliance on high-resolution labeled images: By designing a self-supervised L1 loss function based on reference frame registration and utilizing the downsampled registration error between the low-resolution reference frame and the reconstructed image as the optimization target, this method enables training of super-resolution reconstruction models without the need for high-resolution labeled images. This strategy effectively addresses the reliance on manual annotation and data accessibility inherent in existing deep learning methods, significantly improving the model's practicality, deployability, and cross-scenario generalization capabilities.

[0046] 3. The present invention proposes a displacement-encoded pixel mapping module (DEPM) to improve the accuracy and consistency of feature relocation: After the encoding feature extraction, the present invention introduces a displacement-based DEPM to convert the low-resolution depth feature of each frame into a pixel map. Through displacement Mapping the image onto a high-resolution grid, the spatial repositioning is performed accurately using sub-pixel interpolation. This processing step effectively solves the problems of inter-frame misalignment and fusion blur in existing multi-frame reconstruction methods, thereby improving the accuracy of feature alignment and pixel-level fusion quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of the self-supervised image super-resolution structure based on the active displacement imaging system of the present invention;

[0048] Figure 2 Schematic diagram of the structure of the self-supervisory network based on the active displacement imaging system of the present invention;

[0049] Figure 3 Schematic diagram of the structure of the encoder of the present invention;

[0050] Figure 4 This is a schematic diagram of the structure of the deep channel sensing unit DCAU of the present invention;

[0051] Figure 5 This is a schematic structural diagram of a displacement-encoded pixel mapping module according to the present invention;

[0052] Figure 6Schematic diagram of the structure of the decoder of the present invention;

[0053] Figure 7 The self-supervised loss of the present invention Schematic diagram of the structure;

[0054] Figure 8 This is a comparison diagram of the super-resolution result images generated by the present invention and other reconstruction methods in this embodiment. DETAILED DESCRIPTION

[0055] In this embodiment, a self-supervised image super-resolution method based on an active displacement imaging system achieves high-quality image super-resolution reconstruction through actively controlled sub-pixel displacement and self-supervised learning strategies in the absence of high-resolution labeled images, thereby breaking through the limitations of existing hardware and software strategies in improving image resolution and achieving more accurate and high-quality image reconstruction. Figure 1 As shown in the figure, the active displacement system includes: an image sensor, a host computer processing module, a lens, a controller, and a piezoelectric displacement stage. The image sensor is fixed on the piezoelectric displacement stage, and the center line of the image sensor and the center line of the lens are aligned, which serves as the initial position of the piezoelectric displacement stage. The self-supervised image super-resolution method is performed according to the following steps:

[0056] Step 1: Define the number of frames currently collected as t and initialize t=1;

[0057] After the lens focuses on the object to be measured, the image sensor images the object at the initial position and shifts the captured frame t-1 of the object to the image sensor. As a reference frame Then send it to the host computer processing module and record the initial position coordinates Where H represents the height of the shifted image, W represents the width of the shifted image, and C represents the number of channels of the shifted image. This design ensures that the shifted image has a sufficient number of samples on the sub-pixel grid, providing a stable foundation for subsequent feature alignment and reconstruction.

[0058] Step 2: This system uses a high-precision two-dimensional piezoelectric displacement platform as an active displacement actuator to accurately control the displacement of the image sensor under the condition of fixed lens, thereby realizing data acquisition with sub-pixel relative displacement. After the frame shift image is generated, the t-th displacement instruction is sent to the controller; the controller generates the t-th displacement accuracy parameters in the X and Y directions according to the t-th displacement instruction and transmits them to the piezoelectric displacement stage; the piezoelectric displacement stage moves in the X and Y directions according to the received t-th displacement accuracy parameters. The pixel size is times the step displacement, which drives the image sensor to move for the tth time. After reaching the preset position, the host computer processing module uses the image sensor to collect the tth frame shift image of the observed object. After that, send it to the host computer processing module and record the t-th frame shift image Location coordinates ; Thus, we get an image sequence of T frames ,in, Indicates the magnification factor, represents the total number of shifted images. Compared with traditional methods that rely on natural camera jitter or motion estimation, this active control strategy can accurately capture images at preset positions, effectively avoiding displacement deviations caused by motion estimation errors, and ensuring that the sequence images have highly precise and controllable sub-pixel displacement information.

[0059] Step 3: Use the active displacement control system to accurately record the acquisition position and calculate its displacement information relative to the initial frame. Use formula (1) to calculate the image sequence The t-th frame shift image Relative to the reference frame Inter-frame displacement information :

[0060] (1)

[0061] In formula (1), ( ) represents the relative displacement mapping function;

[0062] Step 4: Construct a self-supervised image super-resolution network based on the active displacement imaging system. The specific structure is as follows: Figure 2 As shown in Figure 1, it includes: encoder, displacement coding pixel mapping module, decoder and self-supervised loss module; First, in the encoding stage, the network performs deep feature extraction on the low-resolution image input for each frame. Figure 3 As shown in the figure, each frame of the image is processed in sequence by a convolutional layer, K cascaded deep channel-aware units (DCAUs) with 64 channels, and a convolutional layer to extract multi-scale features with rich expressive capabilities.

[0063] Using encoder Perform feature extraction and use formula (2) to obtain the depth feature of the t-th frame ; Where N represents the number of channels of the encoder output feature;

[0064] (2)

[0065] In formula (2), represents the convolutional layer, represents the K-th cascaded deep channel perception unit, represents another convolutional layer; Represents the parameters of the encoder, and all inter-frame parameters are shared; the encoder has good modularity and replaceability, and can be flexibly replaced with other mainstream feature extraction networks according to task requirements.

[0066] In this embodiment, each cascaded deep channel perception unit introduces an attention mechanism to model the dependencies between channels and enhance the feature representation capability. The specific structure is as follows: Figure 4 As shown, they all include: several convolutional layers, global pooling layers, and the first fully connected layer , the second fully connected layer And Sigmoid activation function module;

[0067] When k=1, after the input feature j is input into the kth cascade deep channel perception unit, it first passes through several consecutive convolutional layers to extract spatial features and output the convolution feature ; Then pass the global pooling layer to The dimension on each channel is The two-dimensional features are compressed to obtain the scalar value of each channel, thus obtaining the dimension The global vector of the global feature vector is input to the first fully connected layer It is used to compress the channel dimension of the global feature vector to 1 / S times of the original one and output a dimension of Efficient feature representation vector of ;

[0068] Efficient feature representation vector through the second fully connected layer Restore its channel dimension to its original size and output dimension After the channel weight vector is obtained, the channel weight vector is normalized by the sigmoid function to obtain the channel attention weight vector of the kth cascade ; Thus, the deep channel perception output by the k-th cascaded deep channel perception unit is obtained using formula (3) :

[0069] (3)

[0070] When k=2,3,…,K, the deep channel perception output by the k-1th cascaded deep channel perception unit is Input the deep channel perception unit of the kth cascade for processing, and obtain the deep channel perception output by the kth cascade deep channel perception unit , so that the K-th cascaded deep channel perception unit outputs the final deep channel perception .

[0071] Step 5: Figure 5 As shown, the displacement coding pixel mapping module is used to convert the features output by the encoder Mapped to a unified high-resolution grid G, this is the core structure for achieving cross-frame information alignment and fusion. right Perform spatial relocation to obtain high-resolution features of the tth frame , thus obtaining a high-resolution feature map ;

[0072] Step 5.1: Based on the set magnification S, the reference frame Corresponding image size Build a High-resolution grid G; using formula (4) Processing is performed to obtain the high-resolution features of the tth frame , thus obtaining the high-resolution features of T frames , :

[0073] (4)

[0074] In formula (4), Represents the displacement coding pixel mapping module; specifically, according to the inter-frame displacement information , use bilinear interpolation to convert the low-resolution feature map of T frame Each pixel feature in Remap to the target position in the high-resolution grid G , achieving sub-pixel precision feature positioning, thus achieving high-resolution features of T frames .

[0075] Step 5.2, The high-resolution features in the image are aggregated and fused into a unified high-resolution feature map of N-layer feature channels. .

[0076] Step 6: The decoder is responsible for restoring the mapped high-resolution feature map to the final super-resolution image. The specific structure is as follows Figure 6 As shown in Figure 1, it includes: an input convolution layer, multiple DCAUs, and an output convolution layer. To enhance the ability to restore details, the original input features and the features processed by DCAU are added element by element before the output convolution, thereby retaining some low-level semantic information. Reconstruct and obtain super-resolution images ;

[0077] (5)

[0078] In formula (5), Represents the parameters of the decoder; Represents a residual connection. The decoder has good modularity and interchangeability, and can be flexibly replaced with other mainstream feature extraction networks according to task requirements.

[0079] Step 7: In order to make full use of the sub-pixel displacement image sequence collected by the Active Displacement Imaging (ADI) system and avoid the dependence on high-resolution labels, the present invention proposes a self-supervised image super-resolution reconstruction method based on external supervision consistency constraints. Figure 7 As shown in the figure, this method introduces an independently collected supervised reference frame The self-supervised L1 loss is constructed using Equation (6) to construct the self-supervised L1 loss. :

[0080] (6)

[0081] In formula (5), represents the downsampling operation, Obtain a supervision frame independently collected under the same scene for the active displacement system; represents the image registration operation based on displacement compensation, Supervisory frame Displacement compensation in X and Y directions; Represents the L1 norm, which is used as a consistency constraint to ensure both robustness and noise resistance;

[0082] Step 8: Jointly optimize and update the network parameters of the encoder and decoder through the back propagation algorithm; until Until convergence, the optimal image super-resolution reconstruction model is obtained, which is used to super-resolve the displacement images of the collected observation objects.

[0083] Although the supervision frame The images used for loss calculation during training are not fed into the network encoding path. Therefore, the network cannot be Minimizing the loss effectively avoids the problem of trivial solution. In the image acquisition of the present invention, all frames contain independent random noise, which the network cannot reproduce during training. The specific noise structure in the image is unknown, so only an ideal noise-free image can be generated by optimizing the reconstruction process. The final output is downsampled and aligned and used to provide supervision to guide the network to converge to a high-quality solution.

[0084] In the inference phase, since the network has completed parameter optimization through self-supervised learning, there is no need to introduce supervision frames. The present invention only needs to collect S 2 Frame image sequence The input is processed in the trained self-supervised model to obtain a super-resolution image reconstructed with an S-fold magnification; at the same time, the practicality and convenience of the algorithm in actual application scenarios are improved, effectively overcoming the limitation of hardware in obtaining high-resolution labels under specific conditions, and fully demonstrating the advantages of artificial intelligence-enabled instrument imaging. In addition, the self-supervised training strategy proposed in the present invention has good generalization adaptability. As long as the network structure contains an independent feature fusion module, this method can be flexibly extended to a variety of active displacement imaging or multi-view reconstruction tasks, showing a wide range of engineering application potential and scalability.

[0085] The effect of the present invention is further described below in conjunction with experiments:

[0086] During the training process of the proposed self-supervised network, the Adam optimizer is used to update the network parameters, and the initial learning rate is set to 1×10 -4 To ensure stable and efficient model convergence, a periodic learning rate decay strategy was adopted. After every 500 epochs of training, the learning rate decayed to 0.3 times the current value. The training batch size was set to 16, meaning that 16 sets of sequential images were randomly sampled for forward propagation and parameter updates during each iteration. The network was trained end-to-end, lasting approximately 2,500 epochs until the model fully converged.

[0087] In order to measure the performance of the present invention in super-resolution, it is compared with several super-resolution algorithms. The compared methods are divided into two categories: one is the traditional multi-frame super-resolution method, including the projection on convex sets (POCS) and multi-image interpolation; the other is the deep learning method that has performed outstandingly in the field of single-frame super-resolution in recent years, including the dual aggregation transformer (DAT) and the dense-residual-connected transformer (DRCT). In order to verify the effectiveness of the present invention under the condition of a magnification of 4, this example sets the magnification S to 4, thus creating the corresponding low-resolution sequence image S 2It is 16 frames, and twenty numbers with a magnification of ×4 are listed. The peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) results of each method on the Set5, Set14, BSD100, Urban100 and Manga109 standard test sets are shown.

[0088] Table 1. Comparison of quantitative indicators of reconstructed images in public datasets

[0089]

[0090] The comparative analysis results shown in Table 1 clearly demonstrate that the present invention demonstrates significant advantages over other existing image reconstruction techniques in terms of peak signal-to-noise ratio and structural similarity, two key image quality metrics. This demonstrates that the present invention can more efficiently enhance the sharpness of image details and the accuracy of overall structure, thereby producing results that are closer to the original high-resolution image.

[0091] like Figure 8 As shown, when comparing the visual effects of the self-supervised super-resolution reconstruction network based on the active displacement imaging system in the present invention with several other methods, especially when reconstructing low-resolution images at a ×4 magnification, "img096" in the Urban100 dataset is selected as the actual high-resolution image sample, it can be seen that the image reconstructed by the present invention is particularly outstanding in terms of structural integrity and detail clarity. Its reconstruction quality is far superior to other algorithms, and it more accurately reproduces the true high-resolution image features. Although networks such as DAT and DRCT may produce unnatural textures in certain local areas during the single-frame image reconstruction process, the present invention utilizes a sequence of images with different sub-pixel offsets collected by the active displacement system, allowing the self-supervised network model to obtain and use more comprehensive image information, thereby effectively avoiding the occurrence of such problems.

Claims

1. A self-supervised image super-resolution method based on an active displacement imaging system, the active displacement system comprising: Image sensor, host computer processing module, lens, controller and piezoelectric translation stage; The image sensor is fixed on the piezoelectric displacement stage, and the center line of the image sensor is made to coincide with the center line of the lens, which serves as the initial position of the piezoelectric displacement stage. The self-supervised image super-resolution method is characterized in that it is performed according to the following steps: Step 1: Define the number of frames currently collected as t and initialize t=1; After the lens focuses on the object to be measured, the image sensor images the object at the initial position and shifts the acquired frame t-1 of the object to be measured. As a reference frame Then send it to the host computer processing module and record the initial position coordinates ; Among them, H represents the height of the shifted image, W represents the width of the shifted image, and C represents the number of channels of the shifted image Step 2: The host computer processing module receives the After the frame shift image, the t-th displacement instruction is sent to the controller; The controller generates the t-th displacement accuracy parameters in the X direction and the Y direction according to the t-th displacement instruction and transmits them to the piezoelectric displacement stage; The piezoelectric displacement platform is configured to move in the X and Y directions according to the received t-th displacement accuracy parameter. The pixel size is a step displacement, which drives the image sensor to move for the tth time. After reaching the preset position, the host computer processing module uses the image sensor to collect the tth frame shift image of the observed object. After that, send it to the host computer processing module and record the t-th frame shift image Location coordinates ; Thus, we get an image sequence of T frames ,in, Indicates the magnification factor, Indicates the total number of shifted images; Step 3: Use formula (1) to calculate the image sequence The t-th frame shift image Relative to the reference frame Inter-frame displacement information ; (1) In formula (1), ( ) represents the relative displacement mapping function; Step 4: Use the encoder to Perform feature extraction to obtain the depth feature of the tth frame ; Where N represents the number of channels of the encoder output feature; Step 5: Based on the inter-frame displacement information right Perform spatial relocation to obtain high-resolution features of the tth frame , thus obtaining a high-resolution feature map ; Step 6: Use the decoder to Reconstruct and obtain super-resolution images ; Step 7: Use formula (5) to construct self-supervised L1 loss : (6) In formula (6), represents the downsampling operation, Obtaining a supervision frame independently collected under the same scene for the active displacement system; represents the image registration operation based on displacement compensation, Supervisory frame Displacement compensation in X and Y directions; represents the L1 norm; Step 8: Jointly optimize and update the network parameters of the encoder and decoder through the back propagation algorithm; until Until convergence, the optimal image super-resolution reconstruction model is obtained, which is used to super-resolve the displacement images of the collected observation objects.

2. The self-supervised image super-resolution method based on active displacement imaging system according to claim 1, characterized in that: The encoder is obtained using formula (2) : (2) In formula (2), represents the convolutional layer, represents the K-th cascaded deep channel perception unit, represents another convolutional layer; Represents the parameters of the encoder.

3. The self-supervised image super-resolution method based on active displacement imaging system according to claim 1, characterized in that: The step 5 comprises: Step 5.1: Use formula (4) to Processing is performed to obtain the high-resolution features of the tth frame , thus obtaining the high-resolution features of T frames , : (4) In formula (4), Represents the displacement coded pixel mapping module; Step 5.2, The high-resolution features in the image are aggregated and fused into a unified high-resolution feature map. .

4. The self-supervised image super-resolution method based on active displacement imaging system according to claim 3, characterized in that: The displacement coded pixel mapping module in step 5.1 includes: Step 1: Based on the set magnification S, the reference frame Corresponding image size Construct a dimension High-resolution grid G; Step 2: Based on the inter-frame displacement information , use bilinear interpolation to convert the low-resolution feature map of T frame The position of each pixel feature point in Remap to the corresponding pixel feature point target position in the high-resolution grid G , thus obtaining high-resolution features of T frames .

5. The self-supervised image super-resolution method based on active displacement imaging system according to claim 1, characterized in that: The decoder is obtained using formula (5) : (5) In formula (5), Represents the parameters of the decoder; represents a residual connection.

6. The self-supervised image super-resolution method based on active displacement imaging system according to claim 3, characterized in that: Each cascaded deep channel perception unit includes: several convolutional layers, global pooling layers, the first fully connected layer , the second fully connected layer And Sigmoid activation function module; When k=1, after the input feature j is input into the kth cascade deep channel perception unit, it first passes through several consecutive convolutional layers to extract spatial features and output the convolution feature ; Then pass the global pooling layer to The dimension on each channel is The two-dimensional features are compressed to obtain the scalar value of each channel, thus obtaining the dimension The global feature vector is input to the first fully connected layer It is used to compress the channel dimension of the global feature vector to 1 / S times of the original one and output a dimension of Efficient feature representation vector of ; The efficient feature representation vector is passed through the second fully connected layer Restore its channel dimension to its original size and output dimension After the channel weight vector is obtained, the channel weight vector is normalized by the sigmoid function to obtain the channel attention weight vector of the kth cascade ; Thus, the deep channel perception output by the k-th cascaded deep channel perception unit is obtained using formula (3) : (3) When k=2,3,…,K, the deep channel perception output by the k-1th cascaded deep channel perception unit is Input the deep channel perception unit of the kth cascade for processing, and obtain the deep channel perception output by the kth cascade deep channel perception unit , so that the K-th cascaded deep channel perception unit outputs the final deep channel perception .

Images