A mobile terminal-oriented lightweight multi-outlet image super-resolution model construction and deployment method

Abstract

The application provides a mobile terminal-oriented lightweight multi-output image super-resolution model construction and deployment method, which constructs a single image super-resolution network and can select different complexity reconstruction output ends according to different inference requirements. The network comprises a shallow feature extraction module, a progressive deep feature extraction module and a plurality of reconstruction output modules. In the training stage, the plurality of reconstruction output modules jointly participate in joint training; after completing the joint training, the network is subjected to quantization perception training, and a mobile terminal inference model subgraph with a plurality of selectable reconstruction output ends is derived and spliced. In the mobile terminal inference, a target inference mode is determined according to a preset mode selection instruction, a corresponding subgraph path is executed to a target network depth position, and a super-resolution output result is generated through a target reconstruction output end. The application can realize multiple inference modes such as low delay, balance and high quality in a single network, and reduce redundant calculation and model deployment overhead.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, image super-resolution reconstruction, lightweight deep neural networks and mobile deployment technology, and in particular to a method for constructing and deploying a lightweight multi-exit image super-resolution model for mobile devices. Background Technology

[0002] Image super-resolution reconstruction refers to the process of recovering a high-resolution image from a low-resolution image. Its goal is to improve the spatial resolution of the image while restoring as much texture, edge, and detail information as possible. This technology can be applied to scenarios such as mobile image enhancement, video display optimization, image editing, security monitoring, and multimedia content processing.

[0003] In recent years, deep learning-based image super-resolution methods have demonstrated excellent performance in image quality reconstruction. Convolutional neural networks (CNNs) extract features from low-resolution images through multi-layer convolutional structures and then upsample them to generate high-resolution images. Compared to traditional interpolation methods, deep learning methods can recover more image details and achieve better visual results.

[0004] However, deep learning-based super-resolution models often contain numerous convolutional layers and a large number of channels, resulting in a large number of model parameters and computational overhead. Therefore, when deployed on mobile devices, they are easily limited by processor power, RAM, battery consumption, and real-time requirements. Especially on resource-constrained devices such as smartphones and embedded terminals, complex super-resolution models can lead to high inference latency and increased power consumption, making it difficult to meet the needs of real-time image enhancement or interactive applications.

[0005] Addressing the challenges of large parameter counts and computational complexity in deep learning-based super-resolution models has become a research hotspot in the field of image super-resolution. Existing techniques typically employ methods such as lightweight network structures, depthwise separable convolutions, channel pruning, model pruning, parameter sharing, or model quantization to compress the network. While these methods can reduce the number of model parameters and computational complexity to varying degrees, most still rely on a single inference path, meaning the model can only output a super-resolution result with fixed quality and complexity during inference.

[0006] In real-world mobile applications, different scenarios have varying requirements for inference speed, power consumption, and image reconstruction quality. For example, in preview, real-time processing, or low-power scenarios, users prioritize low latency and low power consumption; in static image enhancement or high-quality display scenarios, users prioritize image detail restoration and reconstruction quality. Therefore, super-resolution models relying solely on a single inference path are insufficient to simultaneously adapt to the diverse operating states and user needs of mobile devices.

[0007] Furthermore, super-resolution models require model conversion and quantization when deployed on mobile devices. Inappropriate quantization can introduce significant accuracy loss. Especially for super-resolution networks with multiple reconstruction output modules or multiple inference modes, the network depth and feature distributions corresponding to different reconstruction output modules are different. Direct quantization to int8 may lead to inconsistent image quality degradation under different inference modes, thus affecting the stable use of the model on mobile devices.

[0008] Therefore, it is necessary to provide a lightweight image super-resolution model construction and deployment method suitable for mobile devices, so that a single image super-resolution network can select reconstruction outputs of different complexities according to different inference needs, and reduce the inference overhead on mobile devices while ensuring the image reconstruction quality, thereby improving the flexibility and practicality of the model in mobile deployment. Summary of the Invention

[0009] Objective: To address the problems of existing mobile image super-resolution models having large parameter and computational loads, limited inference modes, and difficulty in flexibly switching image reconstruction quality and inference speed according to different mobile operating requirements, this invention proposes a lightweight, multi-output image super-resolution model construction and deployment method for mobile devices. This invention constructs a single image super-resolution network and sets multiple reconstruction output modules at different network depths within this network. This allows mobile devices to select the corresponding inference mode based on different needs such as low latency, balanced processing, or high-quality reconstruction, thereby reducing inference overhead on mobile devices while ensuring image reconstruction quality.

[0010] To achieve the above technical objectives, this invention proposes a lightweight multi-exit image super-resolution model construction and deployment method for mobile devices, comprising the following steps:

[0011] Construct a single image super-resolution network and complete the initialization of the single image super-resolution network, wherein the single image super-resolution network includes a shallow feature extraction module, a progressive deep feature extraction module, and multiple reconstruction output modules; Acquire a low-resolution image and divide the low-resolution image into low-resolution image blocks of a preset fixed size; The low-resolution image patch is input into the single image super-resolution network, shallow features are extracted by the shallow feature extraction module, and the shallow features are input into the progressive deep feature extraction module to form cumulative features at different network depth positions step by step. The multiple reconstruction output modules are configured based on the accumulated features formed at different network depth locations, so that the multiple reconstruction output modules correspond to different inference modes respectively; During the training phase, multiple super-resolution output results are generated based on the multiple reconstruction output modules, and reconstruction loss and multi-outlet distillation loss are calculated based on the multiple super-resolution output results. A joint training loss is constructed based on the reconstruction loss and the multi-outlet distillation loss, and the network parameters of the single image super-resolution network are updated based on the joint training loss to obtain the jointly trained single image super-resolution network. The single image super-resolution network after joint training is subjected to quantization-aware training. Pseudo-quantization nodes are introduced into the single image super-resolution network, and the joint training loss is calculated based on the super-resolution output results corresponding to the multiple reconstruction output modules to obtain the quantization model. The quantization model is exported as multiple mobile inference model subgraphs, and the multiple mobile inference model subgraphs are spliced ​​together to generate a mobile inference model with multiple selectable reconstruction output modules. During the mobile inference stage, in response to the preset mode selection instruction, the target reconstruction output module and its corresponding target network depth position are determined from the plurality of reconstruction output modules, so that the mobile inference model is executed to the target network depth position, and the super-resolution output result corresponding to the target inference mode is generated through the target reconstruction output module.

[0012] As a preferred embodiment, the low-resolution image block is a three-channel image block of a preset fixed size, and the magnification factor of the single image super-resolution network is a preset integer multiple.

[0013] As a preferred embodiment, the shallow feature extraction module includes at least one convolutional layer for mapping the low-resolution image patch into a shallow feature map.

[0014] As a preferred embodiment, the progressive deep feature extraction module includes multiple stage feature extraction units connected in sequence. The multiple stage feature extraction units pass features step by step along the network depth direction and form accumulated features at different network depth positions for use by the multiple reconstruction output modules. The stage feature extraction unit includes a spatial feature enhancement branch and a channel mapping branch; The spatial feature enhancement branch uses ordinary convolution operations to extract local spatial features; The channel mapping branch uses point convolution operations to perform channel mapping. The features output by the spatial feature enhancement branch, the features output by the channel mapping branch, and the cumulative features of the previous network depth position are fused to obtain the cumulative features of the current network depth position.

[0015] As a preferred embodiment, the plurality of reconstruction output modules include a first reconstruction output module, a second reconstruction output module, and a third reconstruction output module, which correspond to the first inference mode, the second inference mode, and the third inference mode, respectively. The cumulative features corresponding to the second reconstruction output module are obtained by fusing the cumulative features of the first network depth position, the features output by the second stage spatial feature enhancement branch, and the features output by the second stage channel mapping branch. The cumulative features corresponding to the third reconstruction output module are obtained by fusing the cumulative features of the second network depth position, the features of the third stage spatial feature enhancement branch output, and the features of the third stage channel mapping branch output.

[0016] As a preferred embodiment, the first inference mode is a lite low-latency mode, which is used to perform image super-resolution reconstruction when the mobile device is in a state of low latency requirement, low power requirement, or limited computing power. The second inference mode is the mid equalization mode, which is used for image super-resolution reconstruction when the mobile device is in a state of balancing inference speed and reconstruction quality. The third inference mode is the full high-quality mode, which is used for image super-resolution reconstruction when the mobile device has a high image reconstruction quality requirement.

[0017] As a preferred embodiment, the joint training loss includes calculating multiple reconstruction losses corresponding to the multiple reconstruction output modules respectively, and then performing a weighted summation of the multiple reconstruction losses; The joint training loss also includes a multi-outlet distillation loss; the multi-outlet distillation loss is constructed based on the super-resolution output results generated by the first reconstruction output module, the second reconstruction output module, and the third reconstruction output module, and is used to constrain the output differences between different reconstruction output modules.

[0018] As a preferred approach, when exporting the mobile inference model subgraph, the pixel rearrangement operation and pixel value clipping operation are embedded in the model computation graph, or executed as an external post-processing step, depending on the mobile inference framework's support for operators.

[0019] As a preferred embodiment, the quantization-aware training introduces pseudo-quantization nodes into the single image super-resolution network, enabling the single image super-resolution network to update its network parameters under quantization constraints, thereby obtaining a quantization model suitable for mobile inference. The quantization perception training uses low-resolution image patches obtained by cropping low-resolution images according to a preset size as training input samples, and the size, data range, and preprocessing method of the training input samples are consistent with the input requirements of the mobile terminal inference model.

[0020] As a preferred embodiment, the preset mode selection instruction is determined by at least one of user input, preset configuration, device operating status, inference latency requirements, power consumption requirements, or image reconstruction quality requirements.

[0021] Compared with the prior art, the present invention has the following beneficial effects: This invention uses a single image super-resolution network to achieve multiple inference modes, eliminating the need to deploy multiple independent super-resolution models on mobile devices and reducing model deployment and management costs.

[0022] This invention sets up multiple reconstruction output modules at different network depths, enabling mobile devices to select different inference modes based on requirements such as low latency, balanced processing, or high-quality reconstruction, thereby improving the flexibility of the model in mobile applications.

[0023] This invention employs a progressive deep feature extraction module, which passes features step by step through multiple stage feature extraction units and forms cumulative features at different network depths for use by multiple reconstruction output modules, thereby improving the reuse efficiency of cumulative features.

[0024] This invention enhances the feature representation capability under different inference modes by using spatial feature enhancement branches, channel mapping branches, and cumulative feature fusion, enabling the current network depth position to continue to form deeper cumulative features based on the cumulative features of the previous network depth position.

[0025] This invention enables multiple reconstruction output modules to be collaboratively optimized during the training phase through multi-exit joint training and multi-exit distillation loss, which helps to improve the image reconstruction quality under low computational complexity inference mode.

[0026] This invention uses quantization-aware training to generate quantization models suitable for mobile inference, which can reduce model storage overhead and inference computation costs while ensuring accuracy, thereby improving mobile deployment efficiency.

[0027] This invention enables mobile inference models with fixed input sizes to be applied to super-resolution reconstruction of larger images by using fixed-size image blocks and a block-based processing method. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the overall process of building and deploying a lightweight, multi-exit image super-resolution model for mobile devices.

[0029] Figure 2 This is a schematic diagram of the overall structure of a single image super-resolution network.

[0030] Figure 3 This is a schematic diagram of the structure of the stage feature extraction unit.

[0031] Figure 4 This is a flowchart illustrating the process of multi-exit joint training and quantitative perception training. Detailed Implementation

[0032] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0033] like Figure 1 As shown, this embodiment provides a mobile inference flow for a lightweight multi-exit image super-resolution model construction and deployment method for mobile devices. Specifically, a low-resolution image (LR) is acquired and divided into low-resolution image blocks of a preset fixed size; the low-resolution image blocks are input into the mobile inference model; the mobile inference model performs mode selection and inference control according to a preset mode selection instruction to determine the target inference mode among lite low-latency mode, mid equalization mode, or full high-quality mode; according to the target inference mode, the mobile inference model is executed to the corresponding target network depth position, and a raw sub-pixel tensor is output through the target reconstruction output terminal; pixel rearrangement and pixel value cropping operations are performed on the raw sub-pixel tensor to obtain the high-resolution image (HR).

[0034] In this embodiment, the mobile inference model has multiple selectable reconstruction outputs. The mobile inference stage does not need to simultaneously generate lite, mid, and full super-resolution outputs. Instead, it determines the target reconstruction output based on a preset mode selection instruction, and then, after the mobile inference model reaches the corresponding target network depth, outputs the super-resolution result corresponding to the target inference mode.

[0035] The mobile inference model outputs a raw subpixel tensor. This raw subpixel tensor undergoes pixel rearrangement and pixel value cropping operations to obtain the final high-resolution image. The pixel rearrangement and pixel value cropping operations can be embedded within the model's computation graph or performed as post-processing steps outside the model. In this embodiment, the pixel rearrangement and pixel value cropping operations are performed as post-processing steps outside the model.

[0036] like Figure 2 As shown, a single image super-resolution network includes a shallow feature extraction module, a progressive deep feature extraction module, and multiple reconstruction output modules.

[0037] In this embodiment, the low-resolution image block is a three-channel RGB image block with a size of 128×128×3, and the magnification of the single image super-resolution network is 3 times. This size and magnification are only one specific embodiment. In other embodiments, the low-resolution image block may also adopt other preset fixed sizes, and the magnification may also adopt other preset integer magnifications.

[0038] A shallow feature extraction module is used to map low-resolution image patches into shallow feature maps. In one embodiment, the shallow feature extraction module includes at least one convolutional layer, here using a 3×3 convolutional kernel and a non-linear activation function, to extract shallow features from the low-resolution image patch. Let the input low-resolution image patch be... The shallow features output by the shallow feature extraction module are: Then it can be expressed as:

[0039] in, This represents a regular convolution operation with a kernel size of 3×3. This represents a non-linear activation function.

[0040] The progressive deep feature extraction module comprises multiple sequentially connected stage feature extraction units. Each stage feature extraction unit propagates features forward step by step, forming different cumulative features at different network depths for use by the reconstruction output module.

[0041] In one embodiment, the progressive deep feature extraction module includes three stage feature extraction units, which respectively form cumulative features at a first network depth position, a second network depth position, and a third network depth position. The first network depth position corresponds to the lite low-latency mode, the second network depth position corresponds to the mid equalization mode, and the third network depth position corresponds to the full high-quality mode.

[0042] like Figure 3 As shown, the stage feature extraction unit includes a spatial feature enhancement branch and a channel mapping branch. The spatial feature enhancement branch uses ordinary convolution operations to extract local spatial features, while the channel mapping branch uses point convolution operations to perform channel mapping.

[0043] In one embodiment, the spatial feature enhancement branch uses a 3×3 convolutional layer followed by a non-linear activation function; the channel mapping branch uses a 1×1 convolutional layer. The outputs of the spatial feature enhancement branch and the channel mapping branch are fused with the accumulated features from the previous network depth location to obtain the accumulated features from the current network depth location.

[0044] In the first level, the stage feature extraction unit uses shallow features As input, the first spatial branch feature is obtained through the spatial feature enhancement branch. The first channel mapping feature is obtained through the channel mapping branch. The two are then fused to obtain the first cumulative feature. This process can be represented as:

[0045]

[0046]

[0047] In the second level, the stage feature extraction unit uses the first spatial branch features As input, the second spatial branch feature is obtained through the spatial feature enhancement branch. The second channel mapping feature is obtained through the channel mapping branch. and the first cumulative feature Second spatial branching features Second channel mapping features The fusion yields the second cumulative feature , represented as:

[0048]

[0049]

[0050] In the third level, the stage feature extraction unit uses the second spatial branch features. As input, the third spatial branch features are obtained through the spatial feature enhancement branch. The third channel mapping feature is obtained through the channel mapping branch. and the second cumulative feature Third-space branching features and third channel mapping features The fusion yields the third cumulative feature , represented as:

[0051]

[0052]

[0053] The first reconstruction output module, the second reconstruction output module, and the third reconstruction output module can be based on... , and Generate super-resolution output results with different inference depths.

[0054] During the training phase, the input is low-resolution image patches. Simultaneously generate lite output, mid output, and full output. Let the lite output be... The mid output is The full output is Among them, high-resolution images are .

[0055] In one embodiment, the reconstruction loss between the lite output, mid output, and full output and the high-resolution labeled image is calculated separately:

[0056]

[0057]

[0058] In this embodiment, the full output is used as the teacher output. To prevent the teacher output from being negatively influenced by the student output in the distillation loss, a stopping gradient operation is performed on the full output to obtain the teacher output. :

[0059] Based on the teacher's output, calculate the distillation loss corresponding to the mid output and the lite output respectively:

[0060]

[0061] In this embodiment, the total training loss is:

[0062] Where λ is the distillation loss weight coefficient. The lite low-latency mode and mid equalization mode are constrained by the full high-quality mode during the training phase, thereby improving the image reconstruction quality at the output of lite and mid modes. During the joint training phase of the model, multi-exit distillation loss can be introduced; during the quantization-aware training phase, multi-exit distillation loss can be retained or disabled according to the performance interval requirements of each inference mode to maintain the reconstruction quality difference between different inference modes.

[0063] Backpropagation is performed based on the total training loss to update the network parameters of the single-image super-resolution network. After training, a multi-output image super-resolution network with three reconstruction outputs (lite, mid, and full) is obtained.

[0064] After multi-exit joint training is completed, quantization-aware training is performed on the single-image super-resolution network. Quantization-aware training introduces pseudo-quantization nodes into the network, enabling the network to perceive quantization errors during fine-tuning and update network parameters under quantization constraints.

[0065] During quantization-based training, the joint training loss is still calculated using the super-resolution output results corresponding to multiple reconstruction output modules. Therefore, the lite, mid, and full inference modes can all be optimized under quantization constraints, thereby reducing the risk of inconsistent quality degradation at different outputs after quantization.

[0066] After quantization-based perceptual training is completed, the single-image super-resolution network is exported as a mobile inference model subgraph and then stitched together. In this embodiment, the mobile inference model is a TensorFlow Lite model suitable for deployment on Android devices.

[0067] During the mobile inference phase, the mobile application responds to a preset mode selection command and determines the target reconstruction output and its corresponding target network depth from multiple selectable reconstruction outputs. The preset mode selection command can be determined by at least one of user input, preset configuration, device operating status, inference latency requirements, power consumption requirements, or image reconstruction quality requirements.

[0068] When exporting the mobile inference model, pixel rearrangement and pixel clipping operations can be embedded in the model computation graph, or executed as external post-processing steps, depending on the target mobile inference framework's support for operators. In this embodiment, pixel rearrangement and pixel clipping operations are executed as external post-processing steps to generate a high-resolution image corresponding to the target inference mode. Therefore, the mobile inference stage can select the reconstruction output at the corresponding depth based on the target inference mode, without having to generate super-resolution output results for all inference modes simultaneously during each inference iteration.

[0069] For low-resolution images larger than a preset fixed size, they can be divided into multiple low-resolution image blocks, and then input into the mobile inference model for image super-resolution reconstruction.

[0070] In this embodiment, the size of the low-resolution image patch is 128×128×3. The entire low-resolution image can be cropped according to a preset step size to obtain multiple low-resolution image patches. To reduce potential boundary discontinuities during image patch stitching, an overlap region can be set between adjacent low-resolution image patches; in this embodiment, overlap=16.

[0071] After each low-resolution image patch undergoes super-resolution reconstruction, the corresponding high-resolution image patches are stitched together based on their positions in the original low-resolution image. For overlapping regions, cropping, weighted averaging, or other fusion methods can be used to obtain a complete high-resolution image.

[0072] To verify the image reconstruction effect and mobile deployment stability of the method in this embodiment, a single image super-resolution network was tested on the DIV2K image dataset. During testing, the reconstructed image and the high-resolution label image were converted to the YCbCr color space, and the luminance channel was extracted using the MATLAB-stylergb2ycbcr Y-channel calculation formula for evaluation. Peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) were both calculated on the Y-channel, with Y-channel values ​​ranging from [0, 255]. The cropping parameter was set to 3, and the number of test images N was 100.

[0073] In this embodiment, the Float model, the QATKeras model trained with quantization awareness, and the TFLite model composed of subgraphs were tested respectively. The test results are shown in Table 1.

[0074] Table 1 Test Results

[0075] Table 1 shows that in the Float model, the PSNRs for the lite, mid, and full inference modes are 31.0445 dB, 31.2836 dB, and 31.3720 dB, respectively. The overall image reconstruction quality improves with increasing network depth. After quantization-aware training, the QATKeras model maintains relatively stable reconstruction quality across the three inference modes, indicating that quantization-aware training can preserve model accuracy to a certain extent.

[0076] The TFLite model, with its subgraph stitching, differs from the Float model by a maximum of 0.3068 dB, indicating that by stitching together stage subgraphs and reconstructed output subgraphs to form different inference paths, it is still possible to maintain image reconstruction quality close to that of the complete model.

[0077] The experimental results above show that the method in this embodiment can achieve multiple inference modes such as lite low-latency mode, mid equalization mode and full high-quality mode in a single image super-resolution network, and can still maintain relatively stable reconstruction quality after quantization-aware training and TFLite deployment. At the same time, the inference method based on subgraph stitching can support mobile devices to execute to the corresponding network depth position according to the target inference mode, avoiding unnecessary calculations on all outputs during the inference stage.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight multi-exit image super-resolution model construction and deployment method for mobile devices, characterized in that, Includes the following steps: Construct a single image super-resolution network and complete the initialization of the single image super-resolution network, wherein the single image super-resolution network includes a shallow feature extraction module, a progressive deep feature extraction module, and multiple reconstruction output modules; Acquire a low-resolution image and divide the low-resolution image into low-resolution image blocks of a preset fixed size; The low-resolution image patch is input into the single image super-resolution network, shallow features are extracted by the shallow feature extraction module, and the shallow features are input into the progressive deep feature extraction module to form cumulative features at different network depth positions step by step. The multiple reconstruction output modules are configured based on the accumulated features formed at different network depth locations, so that the multiple reconstruction output modules correspond to different inference modes respectively; During the training phase, multiple super-resolution output results are generated based on the multiple reconstruction output modules, and reconstruction loss and multi-outlet distillation loss are calculated based on the multiple super-resolution output results. A joint training loss is constructed based on the reconstruction loss and the multi-outlet distillation loss, and the network parameters of the single image super-resolution network are updated based on the joint training loss to obtain the jointly trained single image super-resolution network. The single image super-resolution network after joint training is subjected to quantization-aware training. Pseudo-quantization nodes are introduced into the single image super-resolution network, and the joint training loss is calculated based on the super-resolution output results corresponding to the multiple reconstruction output modules to obtain the quantization model. The quantization model is exported as multiple mobile inference model subgraphs, and the multiple mobile inference model subgraphs are spliced ​​together to generate a mobile inference model with multiple selectable reconstruction output modules. During the mobile inference stage, in response to the preset mode selection instruction, the target reconstruction output module and its corresponding target network depth position are determined from the plurality of reconstruction output modules, so that the mobile inference model is executed to the target network depth position, and the super-resolution output result corresponding to the target inference mode is generated through the target reconstruction output module.

2. The lightweight multi-exit image super-resolution model construction and deployment method for mobile terminals according to claim 1, characterized in that, The low-resolution image block is a three-channel image block of a preset fixed size, and the magnification of the single image super-resolution network is a preset integer multiple.

3. The lightweight multi-exit image super-resolution model construction and deployment method for mobile terminals according to claim 1, characterized in that, The shallow feature extraction module includes at least one convolutional layer for mapping the low-resolution image patch into a shallow feature map.

4. The lightweight multi-exit image super-resolution model construction and deployment method for mobile terminals according to claim 1, characterized in that, The progressive deep feature extraction module includes multiple stage feature extraction units connected in sequence. The multiple stage feature extraction units pass features step by step along the network depth direction and form cumulative features at different network depth positions for use by the multiple reconstruction output modules. The stage feature extraction unit includes a spatial feature enhancement branch and a channel mapping branch; The spatial feature enhancement branch uses ordinary convolution operations to extract local spatial features; The channel mapping branch uses point convolution operations to perform channel mapping. The features output by the spatial feature enhancement branch, the features output by the channel mapping branch, and the cumulative features of the previous network depth position are fused to obtain the cumulative features of the current network depth position.

5. The lightweight multi-exit image super-resolution model construction and deployment method for mobile terminals according to claim 1, characterized in that, The plurality of reconstruction output modules include a first reconstruction output module, a second reconstruction output module and a third reconstruction output module, which correspond to the first inference mode, the second inference mode and the third inference mode, respectively; The cumulative features corresponding to the second reconstruction output module are obtained by fusing the cumulative features of the first network depth position, the features output by the second stage spatial feature enhancement branch, and the features output by the second stage channel mapping branch. The cumulative features corresponding to the third reconstruction output module are obtained by fusing the cumulative features of the second network depth position, the features of the third stage spatial feature enhancement branch output, and the features of the third stage channel mapping branch output.

6. The lightweight multi-exit image super-resolution model construction and deployment method for mobile terminals according to claim 5, characterized in that, The first inference mode is the lite low-latency mode, which is used to perform image super-resolution reconstruction when the mobile device is in a state of low latency requirement, low power requirement, or limited computing power. The second inference mode is the mid equalization mode, which is used for image super-resolution reconstruction when the mobile device is in a state of balancing inference speed and reconstruction quality. The third inference mode is the full high-quality mode, which is used for image super-resolution reconstruction when the mobile device has a high image reconstruction quality requirement.

7. The method for constructing and deploying a lightweight multi-exit image super-resolution model for mobile devices according to claim 5, characterized in that, The joint training loss includes calculating multiple reconstruction losses corresponding to the multiple reconstruction output modules respectively, and then performing a weighted summation of the multiple reconstruction losses; The joint training loss also includes a multi-outlet distillation loss; the multi-outlet distillation loss is constructed based on the super-resolution output results generated by the first reconstruction output module, the second reconstruction output module, and the third reconstruction output module, and is used to constrain the output differences between different reconstruction output modules.

8. The lightweight multi-exit image super-resolution model construction and deployment method for mobile terminals according to claim 1, characterized in that, When exporting the mobile inference model subgraph, the pixel rearrangement operation and pixel value clipping operation are embedded in the model computation graph, or executed as an external post-processing step, depending on the support of the mobile inference framework for the operators.

9. The method for constructing and deploying a lightweight multi-exit image super-resolution model for mobile devices according to claim 1, characterized in that, The quantization-aware training introduces pseudo-quantization nodes into the single image super-resolution network, enabling the single image super-resolution network to update network parameters under quantization constraints, thereby obtaining a quantization model suitable for mobile inference. The quantization perception training uses low-resolution image patches obtained by cropping low-resolution images according to a preset size as training input samples, and the size, data range, and preprocessing method of the training input samples are consistent with the input requirements of the mobile terminal inference model.

10. The method for constructing and deploying a lightweight multi-exit image super-resolution model for mobile devices according to claim 1, characterized in that, The preset mode selection instruction is determined by at least one of the following: user input, preset configuration, device operating status, inference latency requirements, power consumption requirements, or image reconstruction quality requirements.

Images