Training method of image enhancement model and related method, device, equipment and medium

Abstract

This disclosure provides a training method for an image enhancement model, along with related methods, apparatus, devices, and media. The method includes: inputting an acquired sample image into an image enhancement model; performing deep feature extraction processing on the sample image features corresponding to the sample image based on partial spatial attention and partial channel attention to obtain target image features; and performing image restoration processing based on the target image features to obtain an enhanced image. Partial spatial attention is used for deep feature extraction processing based on partial image features in the feature space, and partial channel attention is used for deep feature extraction processing based on partial image features in the feature channels. The image enhancement model is trained using a target loss value determined based on the difference between the enhanced image corresponding to the sample image and the label image corresponding to the sample image, resulting in a trained image enhancement model. This method can effectively reduce the weight of the image enhancement model.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and more specifically, to a training method and related methods, apparatus, devices and media for an image enhancement model. Background Technology

[0002] Image enhancement is a technique that uses algorithms to improve the visual quality of images or highlight key features. It aims to transform low-quality, degraded images (such as blurry, low-contrast, or noisy versions) into clearer or higher-resolution versions, thereby improving the performance of downstream computer vision tasks. For example, image enhancement techniques can be used to achieve image restoration, recovering high-quality images from degraded images while preserving the original structure and texture details.

[0003] With the emergence of massive amounts of data, deep learning-based techniques have become the mainstream research direction for image enhancement tasks, significantly outperforming traditional optimization methods in various enhancement tasks, such as image super-resolution, image denoising, image deraining, and low-light image enhancement. Benefiting from the bias prior of convolutional neural networks (CNNs), several variants of CNNs have been proposed for image enhancement tasks, providing a reliable data foundation for subsequent image analysis.

[0004] For example, conditional convolutional models use dynamic kernels to achieve content-aware enhancement to address complex global degradation in image enhancement tasks. Furthermore, Transformers are widely used in various image enhancement tasks. These methods all optimize image quality through different adaptive mechanisms; however, they all incur high storage and computational costs. Summary of the Invention

[0005] This disclosure provides a training method for an image enhancement model, as well as related methods, apparatus, devices, and media, which can effectively reduce the parameter size of the image enhancement model, thereby reducing the resource consumption for model storage and computation.

[0006] According to one aspect of this disclosure, a training method for an image enhancement model is provided. The method includes: acquiring a sample image; inputting the sample image into an image enhancement model to perform deep feature extraction processing on sample image features corresponding to the sample image based on partial spatial attention and partial channel attention, obtaining target image features, and performing image restoration processing based on the target image features to obtain an enhanced image of the sample image; wherein, the partial spatial attention is used to perform deep feature extraction processing based on partial image features in the feature space, and the partial channel attention is used to perform deep feature extraction processing based on partial image features in the feature channels, the partial image features being partial features segmented along the channel dimension for the image features to be extracted, the image features to be extracted including sample image features and image features obtained after each feature extraction processing; determining a target loss value based on the difference between the enhanced image corresponding to the sample image and the label image corresponding to the sample image; and training the image enhancement model using the target loss value to obtain a trained image enhancement model.

[0007] According to another aspect of this disclosure, an image enhancement method is provided, the method comprising: acquiring an image to be enhanced; inputting the image to be enhanced into a trained image enhancement model to perform deep feature extraction processing on the image features corresponding to the image to be enhanced based on partial spatial attention and partial channel attention, to obtain target image features, and performing image restoration processing based on the target image features to obtain a target enhanced image after enhancement of the image to be enhanced; wherein the trained image enhancement model is obtained by the above-described image enhancement model training method.

[0008] According to another aspect of this disclosure, a training apparatus for an image enhancement model is provided. The apparatus includes: a sample acquisition module for acquiring sample images; an image processing module for inputting the sample images into the image enhancement model to perform deep feature extraction processing on sample image features corresponding to the sample images based on partial spatial attention and partial channel attention, obtaining target image features, and performing image restoration processing based on the target image features to obtain an enhanced image after the sample images are enhanced; wherein, the partial spatial attention is used to perform deep feature extraction processing based on partial image features in the feature space, and the partial channel attention is used to perform deep feature extraction processing based on partial image features in the feature channels, the partial image features being partial features segmented along the channel dimension for the image features to be extracted, the image features to be extracted including sample image features and image features obtained after each feature extraction processing; a loss determination module for determining a target loss value based on the difference between the enhanced image corresponding to the sample images and the label image corresponding to the sample images; and a model training module for training the image enhancement model using the target loss value to obtain a trained image enhancement model.

[0009] Optionally, the image processing module may include: a shallow feature extraction submodule, used to perform shallow feature extraction processing on the sample image to obtain sample image features; a deep feature extraction submodule, used to perform deep feature extraction processing on the sample image features based on partial spatial attention and partial channel attention to obtain attention image features; and a feature enhancement processing submodule, used to perform feature enhancement processing on the attention image features to obtain target image features.

[0010] Optionally, the deep feature extraction submodule may include: a feature determination unit, used to determine the sample image features as image features to be extracted; a feature extraction unit, used to perform a deep feature extraction process on the image features to be extracted based on partial spatial attention and partial channel attention to obtain intermediate attention image features; and a deep feature unit, used to determine the intermediate attention image features as image features to be extracted when it is detected that the number of deep feature extraction processes has not reached a preset processing threshold, and return to perform a deep feature extraction process on the image features to be extracted based on partial spatial attention and partial channel attention to obtain intermediate attention image features, until the number of deep feature extraction processes reaches the preset processing threshold, and then determine the intermediate attention image features when the number of deep feature extraction processes reaches the preset processing threshold as attention image features.

[0011] Optionally, the feature extraction unit may include: a spatial processing subunit, used to perform deep feature extraction processing on the image features to be extracted based on partial spatial attention, to obtain spatial attention image features; and a channel processing subunit, used to perform deep feature extraction processing on the spatial attention image features based on partial channel attention, to obtain intermediate attention image features.

[0012] Optionally, the spatial processing subunit may include: a spatial normalization component, used to normalize the image features to be extracted to obtain a first spatial normalized feature; a spatial attention component, used to perform deep feature extraction processing on the first spatial normalized feature based on partial spatial attention to obtain partial spatial attention features; a spatial first residual component, used to perform residual connection processing on the partial spatial attention features based on the image features to be extracted, and to normalize the result of the residual connection processing to obtain a second spatial normalized feature; a spatial context component, used to extract context features from the second spatial normalized feature to obtain spatial context image features; and a spatial second residual component, used to perform residual connection processing on the spatial context image features based on the second spatial normalized feature to obtain spatial attention image features.

[0013] Optionally, the spatial attention component may include: a spatial segmentation subcomponent, used to perform channel segmentation processing on the first spatial normalized features to obtain a first part of spatial image features and a second part of spatial image features; wherein the sum of the channel dimensions corresponding to the first part of spatial image features and the channel dimensions corresponding to the second part of spatial image features is the channel dimension corresponding to the first spatial normalized features; a spatial extraction subcomponent, used to perform deep feature extraction processing on the first part of spatial image features based on partial spatial attention to obtain partial spatial self-attention features; a spatial stitching subcomponent, used to perform channel stitching processing based on the partial spatial self-attention features and the second part of spatial image features to obtain spatially stitched image features; and a spatial processing subcomponent, used to perform feature linear processing on the spatially stitched image features to obtain partial spatial attention features.

[0014] Optionally, the spatial extraction sub-component may be specifically used for: performing linear projection processing based on the first part of the spatial image features to obtain spatial query parameters, spatial key parameters, and spatial value parameters; performing spatial window segmentation processing on the spatial query parameters, spatial key parameters, and spatial value parameters respectively to obtain window query parameters, window key parameters, and window value parameters corresponding to each spatial window; performing spatial self-attention processing on each spatial window based on the spatial query parameters, spatial key parameters, and spatial value parameters to obtain window self-attention features; and performing cascade processing on the window self-attention features corresponding to each spatial window to obtain partial spatial self-attention features.

[0015] Optionally, the spatial context component may include: a convolution processing subcomponent, used to perform partial convolution processing on the second spatially normalized features to obtain partial spatial convolution features; a convolution segmentation subcomponent, used to perform channel segmentation processing on the partial spatial convolution features to obtain a first partial spatial convolution feature and a second partial spatial convolution feature; wherein the sum of the channel dimensions corresponding to the first partial spatial convolution feature and the channel dimensions corresponding to the second partial spatial convolution feature is the channel dimension corresponding to the partial spatial convolution feature; a nonlinear subcomponent, used to perform nonlinear processing on the first partial spatial convolution features to obtain partial nonlinear spatial convolution features; and a gating subcomponent, used to perform gating modulation processing based on the second partial spatial convolution features and the partial nonlinear spatial convolution features to obtain spatial context image features.

[0016] Optionally, the convolution processing sub-component can be specifically used to perform channel segmentation processing on the second spatially normalized feature to obtain a first part of convolutional feature and a second part of convolutional feature; wherein, the sum of the channel dimensions corresponding to the first part of convolutional feature and the channel dimensions corresponding to the second part of convolutional feature is the channel dimension corresponding to the second spatially normalized feature; perform partial convolution processing on the first part of convolutional feature to obtain spatial local feature; perform channel concatenation processing on the spatial local feature and the second part of convolutional feature to obtain spatial local concatenated feature; and perform feature linear processing on the spatial local concatenated feature to obtain partial spatial convolutional feature.

[0017] Optionally, the channel processing subunit may include: a channel normalization component, used to normalize the spatial attention image features to obtain a first channel normalized feature; a channel attention component, used to perform deep feature extraction processing on the first channel normalized feature based on partial channel attention to obtain partial channel attention features; a channel first residual component, used to perform residual connection processing on the partial channel attention features based on the spatial attention image features, and normalize the result of the residual connection processing to obtain a second channel normalized feature; a channel context component, used to extract context features from the second channel normalized feature to obtain channel context image features; and a channel second residual component, used to perform residual connection processing on the channel context image features based on the second channel normalized feature to obtain intermediate attention image features.

[0018] Optionally, the channel attention component may include: a channel segmentation subcomponent, used to perform channel segmentation processing on the first channel normalized features to obtain a first part of channel image features and a second part of channel image features; wherein the sum of the channel dimension corresponding to the first part of channel image features and the channel dimension corresponding to the second part of channel image features is the channel dimension corresponding to the first channel normalized features; a channel extraction subcomponent, used to perform deep feature extraction processing on the first part of channel image features based on partial channel attention to obtain partial channel self-attention features; a channel stitching subcomponent, used to perform channel stitching processing based on the partial channel self-attention features and the second part of channel image features to obtain channel stitched image features; and a channel processing subcomponent, used to perform feature linear processing on the channel stitched image features to obtain partial channel attention features.

[0019] Optionally, the channel extraction sub-component may be specifically used for: performing linear projection processing based on the first portion of channel image features to obtain channel query parameters, channel key parameters, and channel value parameters; performing depthwise separable convolution processing on the channel query parameters, channel key parameters, and channel value parameters respectively to obtain corresponding depth channel query parameters, depth channel key parameters, and depth channel value parameters; performing size transformation processing on the depth channel query parameters, depth channel key parameters, and depth channel value parameters respectively to obtain corresponding target channel query parameters, target channel key parameters, and target channel value parameters; and performing channel self-attention processing based on the target channel query parameters, target channel key parameters, and target channel value parameters to obtain partial channel self-attention features.

[0020] Optionally, the image processing module may include: a residual connection submodule, used to perform residual connection processing on the target image features based on the sample image features to obtain connected image features; and an image restoration submodule, used to perform subpixel convolution processing on the connected image features to obtain an enhanced image after the sample image is enhanced.

[0021] According to another aspect of this disclosure, an image enhancement apparatus is provided, comprising: an image acquisition module for acquiring an image to be enhanced; and an image enhancement module for inputting the image to be enhanced into a trained image enhancement model to perform deep feature extraction processing on the image features corresponding to the image to be enhanced based on partial spatial attention and partial channel attention, to obtain target image features, and performing image restoration processing based on the target image features to obtain a target enhanced image after enhancement of the image to be enhanced; wherein the trained image enhancement model is obtained according to the above-described image enhancement model training method.

[0022] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program, wherein the above-described image enhancement model training method and related methods are executed when the computer program is run by a processor.

[0023] According to another aspect of this disclosure, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, which, when invoked by the processor, executes the above-described training method and related methods for the image enhancement model.

[0024] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program stored in a storage medium; a processor of a computer device reads the computer program from the storage medium and executes the computer program, causing the computer device to perform the above-described training method and related methods for the image enhancement model.

[0025] In order to train a deep learning model capable of improving image quality, this embodiment of the present disclosure, after acquiring a sample image, can input the sample image into an image enhancement model. Based on partial spatial attention and partial channel attention, deep feature extraction processing is performed on the sample image features corresponding to the sample image to obtain target image features. Image restoration processing is then performed based on the target image features to obtain an enhanced image with quality greater than the sample image. Specifically, partial spatial attention is used for deep feature extraction processing based on partial image features in the feature space, and partial channel attention is used for deep feature extraction processing based on partial image features in the feature channels. The partial image features are features segmented along the channel dimension of the image features to be extracted. The image features to be extracted include the sample image features and the image features obtained after each feature extraction process. Therefore, when performing further deep feature extraction processing on the sample image features, by applying spatial attention and channel attention to a portion of the image features to be extracted, the weight parameters used for feature extraction in the image enhancement model are reduced, thereby effectively lightweighting the image enhancement model. Furthermore, based on the difference between the augmented image corresponding to the sample image and the label image corresponding to the sample image, the target loss value is determined, and the image augmentation model is trained using the target loss value to obtain the trained image augmentation model.

[0026] Compared to existing techniques that directly extract features from the entire sample image, which results in a large model parameter size and low computational efficiency due to feature extraction across all feature dimensions, this embodiment of the present disclosure teaches the image enhancement model to perform deep feature extraction based on partial spatial attention and partial channel attention, focusing only on a subset of image features across a portion of the channel dimensions. This reduces the weight parameters used for feature extraction, making the image enhancement model lightweight. Because the image enhancement model, by jointly utilizing partial spatial attention and partial channel attention to extract features from a subset of image features, can learn to locate key regions in the feature space dimension and select key feature types corresponding to those key regions in the feature channel dimension, it achieves accurate feature extraction. Therefore, even with a reduced model parameter size, this embodiment of the present disclosure can still extract accurate image features, ensuring that the performance of the lightweight model in performing image enhancement tasks is not affected. Furthermore, since the embodiments of this disclosure have lightweighted the image enhancement model, that is, reduced the model parameter size of the image enhancement model, the embodiments of this disclosure can effectively reduce the consumption of storage resources when deploying the image enhancement model on a computer device, and effectively reduce the amount of computing resources involved in the calculation of the image enhancement model when performing image enhancement tasks, thereby improving the computing efficiency of the computer device.

[0027] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This diagram illustrates a system architecture provided by an embodiment of the present disclosure.

[0030] Figure 2 This diagram illustrates a system deployment according to an embodiment of the present disclosure.

[0031] Figure 3 This illustration shows an application scenario diagram of a medical diagnostic scenario provided by an embodiment of the present disclosure.

[0032] Figure 4 This diagram illustrates an application scenario of an image restoration work provided by an embodiment of the present disclosure.

[0033] Figure 5 This diagram illustrates an application scenario of autonomous driving provided by an embodiment of the present disclosure.

[0034] Figure 6 This diagram illustrates an application scenario of an industrial inspection scenario provided by an embodiment of the present disclosure.

[0035] Figure 7 This diagram illustrates an application scenario of a security protection scenario provided by an embodiment of the present disclosure.

[0036] Figure 8 This diagram illustrates a flowchart of a training method for an image enhancement model provided in an embodiment of the present disclosure.

[0037] Figure 9 This diagram illustrates a model framework diagram of an image enhancement model provided in an embodiment of this disclosure.

[0038] Figure 10 This diagram illustrates the computation of a partial operator provided in an embodiment of the present disclosure.

[0039] Figure 11 This diagram illustrates the spatial and channel features of a sample image provided in an embodiment of the present disclosure.

[0040] Figure 12 This illustration shows a flowchart of alternating partial attention processing provided by an embodiment of the present disclosure.

[0041] Figure 13 A flowchart illustrating a partial attention processing method provided in an embodiment of this disclosure is shown.

[0042] Figure 14 This diagram illustrates a partial spatial attention block structure provided in an embodiment of the present disclosure.

[0043] Figure 15 This diagram illustrates a partial spatial attention processing method provided in an embodiment of the present disclosure.

[0044] Figure 16 This illustration shows a schematic diagram of spatial window self-attention provided in an embodiment of the present disclosure.

[0045] Figure 17 This diagram illustrates the structure of a partial gating network provided in an embodiment of the present disclosure.

[0046] Figure 18 This diagram illustrates the structure of a partial channel attention block according to an embodiment of the present disclosure.

[0047] Figure 19 This illustration shows an attention diagram of a channel window self-attention provided in an embodiment of the present disclosure.

[0048] Figure 20 This illustration shows an attention diagram of a channel window self-attention provided in an embodiment of the present disclosure.

[0049] Figure 21 This illustration shows a training and testing flowchart provided by an embodiment of the present disclosure.

[0050] Figure 22 This diagram illustrates the quantitative results of an image super-resolution task provided by an embodiment of this disclosure.

[0051] Figure 23 This diagram illustrates the qualitative results of an image super-resolution task provided in an embodiment of this disclosure.

[0052] Figure 24 This diagram illustrates the quantitative results of an image denoising task provided by an embodiment of the present disclosure.

[0053] Figure 25 This diagram illustrates the qualitative results of an image denoising task provided by an embodiment of this disclosure.

[0054] Figure 26 This diagram illustrates the quantitative results of an image deraining task provided by an embodiment of this disclosure.

[0055] Figure 27 The image shown is a qualitative result diagram of an image deraining task provided by an embodiment of this disclosure.

[0056] Figure 28 This diagram illustrates the quantitative results of a low-light image enhancement task provided by an embodiment of the present disclosure.

[0057] Figure 29 This diagram illustrates the qualitative results of a low-light image enhancement task provided by an embodiment of this disclosure.

[0058] Figure 30 The diagram shows a flowchart of an image enhancement method provided in an embodiment of this disclosure.

[0059] Figure 31 This diagram illustrates a deep feature extraction process for an attention block provided in an embodiment of this disclosure.

[0060] Figure 32 This is a block diagram of a training device for an image enhancement model provided in an embodiment of this disclosure.

[0061] Figure 33 This is a block diagram of an image enhancement device provided in an embodiment of this disclosure.

[0062] Figure 34This is a block diagram of a computer device provided in an embodiment of the present disclosure.

[0063] Figure 35 This is a block diagram of a computer-readable storage medium provided in an embodiment of this disclosure. Detailed Implementation

[0064] To enable those skilled in the art to better understand the solutions disclosed herein, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. The implementation methods described with reference to the accompanying drawings are exemplary and are only used to explain this disclosure, and should not be construed as limiting this disclosure. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0065] In some processes described in the specification, claims, and the foregoing drawings, multiple steps are included in a specific order. However, it should be clearly understood that these steps may not be performed in the order they appear herein, or may be performed in parallel. The step numbers are merely used to distinguish different steps and do not themselves represent any execution order. Furthermore, descriptions such as "first," "second," or "objective" in this document are used to distinguish similar objects and are not necessarily used to describe a specific order, sequence, or quantity.

[0066] It is worth noting that in the specific embodiments of this disclosure, data such as the image to be enhanced is involved. When the above embodiments of this disclosure are applied to specific products or technologies, permission or consent from the target is required, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards. For example, when an embodiment of this disclosure needs to obtain the image to be enhanced, separate permission or consent for the image to be enhanced can be obtained through pop-ups or redirection to a confirmation page. After obtaining separate permission or consent, the necessary related data for enabling the embodiments of this disclosure to operate normally can then be obtained.

[0067] Image enhancement is commonly used to transform blurry, low-quality images into clear, high-quality ones. Examples include image restoration tasks. With the advent of massive amounts of data, deep learning-based techniques have become the mainstream research direction for image enhancement, such as image super-resolution, image denoising, image deraining, and low-light image enhancement. Benefiting from the bias priors of convolutional neural networks (CNNs), a series of CNN variants have been proposed for image enhancement tasks, providing a reliable data foundation for subsequent image analysis. For example, the CConv and PConv models optimize image quality through different adaptive mechanisms. Furthermore, existing techniques also focus on the design of large-scale Transformer network architectures to achieve better performance in visual tasks.

[0068] However, the network architecture design of these methods is prone to feature redundancy, resulting in high storage costs and computational overhead. To address this, many effective solutions have been proposed to reduce feature redundancy and accelerate inference. For example, SPConv proposes dividing features into representative and uncertain redundant parts to extract intrinsic information. In contrast, the GhostNet series achieves significant efficiency improvements by generating redundant features through low-cost operations. Spatial and channel redundancy in convolutional layers are reduced jointly through spatial and channel reconstruction. However, these methods primarily analyze convolutional neural networks and do not fully investigate Transformer-based architectures. With the expanding application of deep learning in mobile devices, IoT systems, and edge computing, designing lightweight and efficient deep learning models is crucial for reducing storage costs and computational overhead.

[0069] To address the aforementioned problems, embodiments of this disclosure propose a training method and related methods for image enhancement models. The system implementation environment and related application scenarios of the methods involved in this disclosure will be described below.

[0070] Please see Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The image enhancement model training method and related methods provided in this disclosure can be applied in system 100. The data acquisition device 110 is used to acquire a large number of training samples, i.e., sample images to be enhanced. Each sample image has a corresponding label image (Ground Truth). The quality of the sample images is relatively lower than that of the label images; for example, the sample images are low-resolution images, while the label images can be relatively high-resolution images. The data acquisition device 110 can store the training samples in a database 120.

[0071] The training device 130 can train a preset text elimination model based on training samples in the database 120 to obtain a trained image enhancement model, i.e., the target model 101. The model training method can be supervised learning or similar. The target model 101 can consist of one or more neural networks. For example, the image enhancement model of the target model 101 can be a deep neural network model composed of an encoder, convolutional neural networks (CNNs), a transformer, and a decoder; the model framework is not limited here.

[0072] The execution device 140 can deploy the target model 101 trained by the training device 130. During the computation and other related processing performed by the processing module 141 of the execution device 140, the execution device 140 can call data, programs, etc. in the data storage system 142 for corresponding computational processing, and store the processing results and instructions obtained from the computational processing into the data storage system 142. Data transmission between the execution device 140 and the user device 150 can be performed through an input / output (I / O) interface. For example, when the execution device 140 receives video data containing text to be removed (i.e., a sequence of video frames to be removed) sent by the user device 150, the execution device 140 can call the trained image enhancement model to perform text removal processing on the video frame sequence to be removed, and return the reconstructed video frame sequence after text removal, i.e., the video data with the text removed, to the user device 150.

[0073] The aforementioned training device 130, execution device 140, and user device 150 can be computer devices such as servers or terminals. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), blockchain, and big data and artificial intelligence platforms. The terminal can be a laptop, tablet, desktop computer, etc.

[0074] It should be noted that, Figure 1 This is merely a schematic diagram of a system architecture provided by an embodiment of this disclosure. The system architecture described in this disclosure is for the purpose of more clearly illustrating the technical solutions of this disclosure and does not constitute a limitation on the technical solutions provided by this disclosure. For example, Figure 1The execution device 140 can be used to deploy the trained target model 101. In other cases, the user device 150 can also directly deploy the trained target model 101 and perform text elimination processing locally.

[0075] Please see Figure 2 , Figure 2 A schematic diagram of a system deployment is shown. Exemplarily, the training method and related methods of the image enhancement model provided in this disclosure embodiment can also be deployed in, for example, Figure 2 The system shown is a C / S (Client / Server) architecture system. This system may include servers, gateways, the internet, and computer equipment.

[0076] A server is a computer system that provides certain services to the server-side. Compared to ordinary computer equipment, servers have higher requirements in terms of stability, security, and performance. Figure 2 As shown, server 210 can be a single high-performance computer, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines) within a network platform. A gateway is also known as an internetwork connector or protocol converter. For example... Figure 2 As shown, gateway 220 achieves network interconnection at the transport layer via Internet 230, and is a computer system or device that acts as a translator. Between two systems using different communication protocols, data formats, or languages, or even completely different architectures, a gateway is a translator. At the same time, a gateway can also provide filtering and security functions.

[0077] Computer devices can send requests to servers, requesting access to or use resources, data, or services on the server. For example... Figure 2 As shown, the computer device may include various forms such as a smartphone 240a, a laptop 240b, an in-vehicle terminal 240c, and a personal digital assistant (not identified). Furthermore, it can be a single computer device or a collection of multiple computer devices. The computer device can communicate with the Internet 230 via wired or wireless means. For example, smartphone 240a can communicate with the Internet 230 through a wireless router 250; similarly, messages sent by laptop 240b to server 210 can be transmitted to the corresponding server 210 via gateway 220. Messages sent by server 210 can also be transmitted to in-vehicle terminal 240c via gateway 220.

[0078] In some embodiments, the training method and image enhancement method of the image enhancement model provided in this disclosure can be implemented partly on a server and partly on a computer device. For example, the server can acquire sample images and train an image enhancement model based on the sample images, and the computer device can download and install the image enhancement model from the server for image enhancement.

[0079] It should be noted that, Figure 2 This is merely a schematic diagram of a system deployment provided by an embodiment of this disclosure. The system deployment scheme described in this disclosure is only for the purpose of more clearly illustrating the technical solutions of this disclosure and does not constitute a limitation on the technical solutions provided by this disclosure. For example, in some other embodiments, the training method and related methods of the image enhancement model provided by this disclosure can both be implemented on a computer device, that is, the training steps of the text elimination model and the related steps of text elimination are completed locally by the computer device. As those skilled in the art will recognize, with the evolution of system deployment schemes, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0080] The training method and related methods of the image enhancement model disclosed herein can be applied in various scenarios, including medical diagnosis, image restoration, autonomous driving, industrial inspection, and security applications that require image enhancement tasks. Figure 3 The medical diagnostic scenario shown Figure 4 The image restoration scene shown Figure 5 The autonomous driving scenario shown Figure 6 The industrial inspection scenario shown and Figure 7 The security scenario shown is an example.

[0081] (1) Medical diagnosis scenario.

[0082] In medical diagnosis, the quality of pathological slide images and medical imaging (such as X-rays, computed tomography scans, and magnetic resonance imaging) is crucial for doctors to accurately assess a patient's condition. However, due to factors such as equipment performance and individual patient differences, these medical imaging images and pathological slide images may suffer from noise, insufficient contrast, and other problems. The image enhancement model provided in this disclosure can be used to enhance the details and contrast of the aforementioned images, assisting pathologists in making more accurate diagnoses and studies, and improving diagnostic accuracy. For example, an auxiliary diagnostic service provider can enable an auxiliary diagnostic system based on the training method and related methods of the image enhancement model provided in this disclosure. Furthermore, medical institutions can download the auxiliary diagnostic system from the auxiliary diagnostic service provider's cloud server via the network and install it locally for clinical auxiliary diagnosis.

[0083] like Figure 3As shown, taking tumor auxiliary diagnosis as an example, after a patient completes a lung MRI examination, the MRI system 310 can transmit the patient's MRI image to the auxiliary diagnostic system 320 of the terminal device. The auxiliary diagnostic system 320 then inputs the original image of the patient's corresponding lung MRI image into an image enhancement model for image enhancement processing. That is, by first extracting shallow image features from the original image, then extracting deep image features based on partial spatial attention and partial channel attention, and finally restoring the image, the original image is processed for denoising, super-resolution, and artifact removal. Denoising can preserve the details of the lung structure in the image while reducing interference and improving the signal-to-noise ratio; super-resolution can improve the image resolution, making small lung lesions easier to observe; contrast enhancement can strengthen the grayscale differences between different lung tissues and lesions; artifact removal can reduce visual interference and prevent misdiagnosis. Thus, an enhanced image of the lung MRI image is obtained. In this way, clinicians can view diagnostic instructions on the interactive interface of the auxiliary diagnostic system 320, as well as the original and enhanced images of the patient's lung MRI, for comprehensive medical diagnosis.

[0084] (2) Image restoration scenario.

[0085] Image restoration systems can be developed based on the training methods and related methods for the image enhancement model provided in the embodiments of this disclosure. For example... Figure 4 As shown, the image restoration system may include a server 411 deployed on a cloud server 410 and a client 421 (image restoration APP) deployed on a smartphone 420. The server 411 is primarily responsible for performing complex calculations and logical processing, processing the data requested by the client 421 on the smartphone 420. Typically, the cloud server 410 can train a text removal model based on training data and deploy the server 411 of the image restoration system developed based on the text removal model. Thus, when the smartphone 420 sends an image restoration request to the cloud server 410, the cloud server 410 can invoke the text removal model to perform image enhancement tasks to restore the image and return the restored target image.

[0086] For example, a user can download and install an image repair app as client 421 on a smartphone 420, and open the image repair function interface 4211 of client 421. Figure 4As shown, the user can click the shooting frame to capture the image that needs to be repaired, i.e., the image to be repaired, and then click the upload image button to upload the image to be repaired. After capturing or uploading, the client 421 can send the image to be repaired to the server 411 of the cloud server 410. The server 411 can then call the image enhancement model to enhance the image to be repaired according to the image enhancement method disclosed herein, that is, enhance the blurry image to be repaired into a high-resolution, clear target image, and return the target image to the smartphone 420 for the client 421 to display.

[0087] (3) Autonomous driving scenario.

[0088] In autonomous driving scenarios, enhancing blurred images captured in foggy weather can significantly improve the accuracy of target detection, thereby enhancing the safety of autonomous driving. Autonomous driving systems can be developed based on the image enhancement model training method and related methods provided in this disclosure. Figure 5 As shown, the autonomous driving system 520 is deployed on the vehicle terminal 510. This is because autonomous driving requires millisecond-level response speeds (such as emergency braking and obstacle avoidance). Local deployment can avoid network latency, and the vehicle terminal 510 can directly process sensor data (cameras, lidar, etc.) to ensure the immediacy of control commands.

[0089] During vehicle operation, the autonomous driving system 520 can utilize cameras to capture real-time road condition images to generate driving strategies. For example... Figure 5 As shown, during vehicle operation in foggy or hazy weather, the autonomous driving system 520 can call the system to collect real-time road condition images and transmit them to the image enhancement module 521. The image enhancement module 521 can then call the image enhancement model to enhance the road condition images, resulting in clear enhanced road condition images. Furthermore, the autonomous driving system 520 can generate driving strategies based on the enhanced road condition images to control the vehicle for autonomous driving.

[0090] (4) Industrial testing scenarios.

[0091] like Figure 6The diagram illustrates a product quality inspection scenario in an industrial inspection environment. Product manufacturers can develop a product quality inspection system based on the image enhancement model of this embodiment in actual product manufacturing scenarios. This system can be deployed on a cloud server 610 to inspect the appearance, dimensions, and defects of product 621. Product 621 may include electronic chips, metal parts (gears, bearings, surgical instruments), etc. Specifically, the product quality inspection device 620 can acquire original images of product 621 using cameras 622 mounted at multiple angles. Further, the product quality inspection device 620 can send these original images to the cloud server 610, which has the image enhancement model deployed on it. The cloud server 610 then uses the image enhancement model to enhance the original product image, resulting in an enhanced product image. The cloud server 610 can return this enhanced product image to the product quality inspection device 620, which then uses it to inspect the appearance, dimensions, and defects of product 621, thereby improving the accuracy and efficiency of product inspection.

[0092] (5) Security and prevention scenarios.

[0093] In security detection systems, due to poor lighting conditions (such as at night or on cloudy days) and limitations in camera resolution, the detected images may suffer from problems such as blurriness, excessive noise, and low contrast. Image enhancement technology can process these images in real time, improving image clarity, brightness, and contrast, enabling the security detection system to more accurately identify target personnel. A security detection system developed based on the image enhancement model of this disclosure is deployed on a cloud server 710. During facial recognition access control, the camera of the access control device 720 can capture the original facial image of the visitor. Further, the access control device 720 can send the original facial image to the cloud server 710, where the cloud server 710 calls the image enhancement model to enhance the original facial image, obtaining an enhanced facial image. The cloud server 710 can then return this enhanced facial image to the access control device 720, which performs identity verification based on the enhanced facial image to confirm the visitor's identity.

[0094] It should be noted that, Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7These are merely illustrative diagrams illustrating five application scenarios provided in this disclosure. The application scenarios described in this disclosure are only for clearer explanation of the technical solutions provided in this disclosure and do not constitute a limitation on the technical solutions provided in this disclosure. The training methods and related methods of the image enhancement model provided in this disclosure can also be applied to other application scenarios requiring image enhancement, such as object detection and image recognition. As those skilled in the art will recognize, with the evolution of application scenarios, the technical solutions provided in this invention are equally applicable to similar technical problems.

[0095] According to one embodiment of this disclosure, a method for training an image enhancement model is provided. This method can be executed by a computer device (e.g., a desktop computer, laptop computer, or other terminal device with certain processing capabilities), which at least has storage, computing, and communication functions. Figure 8 A flowchart illustrating the training method of the image enhancement model provided in this embodiment is shown. Figure 8 As shown, the training method for this image enhancement model can specifically include:

[0096] Step 110: Obtain sample images.

[0097] In this embodiment of the disclosure, a sample image refers to an image used as a training sample. Here, H and W represent the height and width of the sample image, respectively, and 3 indicates that the sample image has three color (RGB) channels. The sample image is used as input to the model to be trained (i.e., the image enhancement model) for image enhancement. Image enhancement refers to processing the original image to improve its quality in visual tasks. The goals of image enhancement can include highlighting features of interest, improving the visual appeal of the image, or enhancing the performance of subsequent image processing tasks (such as recognition, detection, and segmentation). For example, the image enhancement model can be used for contrast enhancement, that is, adjusting the contrast relationship between different gray levels in the image. This way, compared to the darker and brighter areas of the sample image, the enhanced image displays darker pixels in darker areas and brighter pixels in brighter areas, thus enhancing the visual contrast of the image. The image enhancement model can also be used for color correction, for example, adjusting the white balance of the sample image or enhancing the color saturation of the sample image, making the enhanced image more vibrant and rich in color. The image enhancement model can also be used for image denoising to reduce noise points in the sample image, resulting in a less noisy enhanced image. Image augmentation models can also be used for super-resolution reconstruction, recovering high-resolution augmented images from low-resolution sample images.

[0098] As an example, the methods for obtaining sample images can include obtaining them from databases, the internet, or terminals, and are not limited here. For example, obtaining them from a database refers to retrieving training samples from a database specifically used to store training samples, which may be deployed on a server. Obtaining them from the internet refers to searching for images on the internet and preprocessing them to obtain sample images that can be directly used for model training. Alternatively, sample images can be obtained directly from publicly available datasets on the internet, such as the DIV2K (DIVerse 2K ResolutionImage Dataset), a high-quality dataset designed for image super-resolution (SR) and other low-level vision tasks. Obtaining them from a terminal refers to preprocessing images acquired in real-time by the terminal to obtain sample images that can be directly used for model training.

[0099] Step 120: Input the sample image into the image enhancement model, and perform deep feature extraction processing on the sample image features corresponding to the sample image based on partial spatial attention and partial channel attention to obtain the target image features. Then, perform image restoration processing based on the target image features to obtain the enhanced image after sample image enhancement.

[0100] The image enhancement model provided in this disclosure is a deep learning model based on the Transformer architecture for performing image enhancement. This model includes three parts: a shallow feature extraction part, a deep feature extraction part, and an image restoration part. For example... Figure 9 The image enhancement model shown is a model framework diagram. The shallow feature extraction part of the image enhancement model can be a convolutional layer, the deep feature extraction part can be composed of N lightweight partial Transformer layers and a convolutional layer, and the image restoration part is composed of sub-pixel convolutional layers.

[0101] The shallow feature extraction section transforms the input sample image from its original pixel space to a feature space, extracting basic and localized image features (such as edges, colors, and textures) during this process—the sample image features. The deep feature extraction section performs further feature mining on the sample image features to obtain richer image features for image enhancement. The image restoration section restores the image features output from the deep feature extraction section to the pixel space, resulting in the enhanced image.

[0102] The partial Transformer layers in the deep feature extraction part consist of partial spatial attention blocks (PSABs) and partial channel attention blocks (PCABs). The PSABs are used to model long-range dependencies of image features in the spatial dimension, while the PCABs are used to model long-range dependencies of image features in the channel dimension. The number N of partial Transformer layers can be set based on the requirements of the actual application scenario, combined with empirical and experimental values.

[0103] Currently, whether based on convolutional neural networks or deep learning models built on the Transformer architecture, image enhancement techniques exhibit significant feature redundancy, meaning that highly correlated or repetitive information exists between different image features. This feature redundancy arises because, to improve model performance, researchers often increase network depth. For example, large-scale Transformers typically stack numerous neural layers. However, excessively deep networks cause features to be repeatedly processed during propagation, resulting in smaller differences between image features extracted by many neural layers, thus generating redundant intermediate features.

[0104] Because model parameters that generate feature redundancy in deep learning models still need to be stored independently—for example, if a Transformer layer has 12 self-attention heads, and 6 of them have similar weights—approximately 50% of the GPU memory is occupied by redundant parameters. Furthermore, features from adjacent layers that generate feature redundancy in a deep learning model may be highly similar, but all intermediate features still need to be cached in GPU memory during backpropagation, leading to wasted GPU memory. Consequently, some hardware resources (computing power) of the computer device are wasted on computations related to feature redundancy, rather than being applied to the extraction of effective features, thus reducing computational efficiency.

[0105] Considering that each neural layer in a deep learning model performing image enhancement tasks typically performs full computation on the input features, but many of these features are redundant or similar, computationally wasting resources would be detrimental. Therefore, this disclosure proposes a partial operator computation method and instantiates this partial operator in the deep feature extraction process of the image enhancement model. Specifically, it includes partial spatial attention, partial channel attention, and convolutional computations within partial spatial attention and partial channel attention. Partial spatial attention is used for deep feature extraction based on partial image features in the feature space, while partial channel attention is used for deep feature extraction based on partial image features in the feature channels. The partial image features are features segmented along the channel dimension of the image features to be extracted, including sample image features and image features obtained after each feature extraction process.

[0106] For easier understanding, please refer to Figure 10 The diagram illustrates the computation of a partial operator, targeting the input features to be processed for deep feature extraction, i.e., the image features to be extracted. First, extract the image feature F along the channel dimension of this feature. in Channel segmentation is performed to obtain the first part of the image features. Second part of image features Where C is the image feature F to be extracted in The channel dimension, C p The first part of the image features F p The channel dimension, C r For the second part of the image features F r The channel dimension, C p +C r =C, the channel segmentation process is random, that is, C p ∈(0,C), for example,

[0107] Furthermore, for the first part of the image features F p Deep feature extraction is performed, and the extracted features are compared with the second part of the image features F. r Channel concatenation is performed, and then the concatenated features are input into a linear layer for linear projection to obtain the output features. The calculation process of some operators is shown in formula (1).

[0108]

[0109] Where CS(·) represents channel splitting processing, This indicates channel concatenation, and W represents the projection matrix corresponding to the linear layer. Operator(·) indicates the instantiation of a partial operator during deep feature extraction. For example, when a partial operator is instantiated as partial spatial attention, Operator(·) is Spatial Window Self-Attention (SWSA); when a partial operator is instantiated as partial channel attention, Operator(·) is Channel-Wise Self-Attention (CWSA); and when a partial operator is instantiated as partial convolution, Operator(·) is specified as convolution.

[0110] As can be seen from the above analysis, the partial operators proposed in this disclosure, during the deep feature extraction process of the image enhancement model, instantiate partial spatial attention and partial channel attention, so that the image enhancement model only extracts features from a portion of the image features in a portion of the channel dimensions. This significantly reduces feature redundancy. Since only a portion of the image features need to be further processed for feature extraction, the scale of the weight parameters used for feature extraction in the image enhancement model is also reduced, thereby effectively lightweighting the image enhancement model, reducing the consumption of computational resources, and improving computational efficiency.

[0111] In particular, with the expanding application of deep learning for image enhancement tasks in mobile devices, IoT systems, and edge computing, this disclosure presents a lightweight and efficient image enhancement model based on partial operator settings. When performing image enhancement tasks on computing devices with limited resources, this model can effectively reduce storage resource consumption and improve computational efficiency. The following will combine... Figure 9 The model framework diagram shown provides a detailed explanation of the process by which the image enhancement model extracts features from the target image and generates the enhanced image.

[0112] In some embodiments, a sample image is input into an image enhancement model, which can perform shallow feature extraction on the sample image to obtain sample image features. Based on partial spatial attention and partial channel attention, deep feature extraction is performed on the sample image features to obtain attention image features. Then, feature enhancement processing is performed on the attention image features to obtain target image features.

[0113] When an image enhancement model receives an input sample image, it first needs to transform the sample image, which is in pixel space, into a feature space (also called vector space or embedding space) to obtain structured data features of the image, i.e., sample image features, making them easier to learn in the model. As an example, shallow feature extraction processing of the sample image can be performed using a convolutional layer to obtain the sample image features. For example, this convolutional layer can be a 3×3 convolutional layer, which extracts the sample image... The input is fed into this convolutional layer, which extracts basic and localized image information from the sample image I to obtain the sample image features.

[0114] Shallow feature extraction processing yields shallow image features corresponding to the sample image, also known as sample image features. These features represent shallow image information such as local details (e.g., edges, colors, textures) and basic structures within the sample image. While this shallow image information is helpful in representing the basic features of an image, it is insufficient to represent complex image semantics and meet the requirements for image enhancement. Therefore, deeper image feature extraction, or deep feature extraction processing, can be performed based on the sample image features.

[0115] In this embodiment, the goal of deep feature extraction processing is to efficiently extract image features that accurately represent the semantics of the image in order to reconstruct a high-quality image. The attention mechanism provides global modeling and dynamic feature interaction capabilities for image feature extraction; the establishment of this global relationship is particularly important for high-quality image reconstruction. Furthermore, in computer vision tasks, image features typically contain spatial and channel information: spatial information reflects the spatial positional relationships between different pixels or regions on an image, preserving the local and global structural information of the image (such as different regions of a painting), while channel information reflects the interdependent relationships between different semantics (such as color and texture). Figure 11 The diagram shown illustrates the spatial and channel features of the sample image. The corresponding spatial dimension is H×W, and the channel dimension is C.

[0116] Spatial attention models long-range dependencies of image features along the spatial dimension, focusing on the importance of different spatial locations within image features—that is, "which regions need to be focused on." Channel attention models long-range dependencies of image features along the channel dimension, focusing on the importance of different feature channels within image features—that is, "which feature types are more important to the current task." If only spatial attention is used, the model may overemphasize certain regions while ignoring the importance of feature types within those regions. For example, although the model may focus on the cat's head, the color channels in that region might be affected by noise. If only channel attention is used, the model may know that "texture is important," but it cannot determine where the texture appears in the image. For example, increasing the weights of the texture channel, if the texture is scattered throughout the image (such as interfering textures in the background), may actually introduce noise.

[0117] To address this, this disclosure proposes a complementary approach, utilizing spatial attention (focusing on "where is important") and channel attention (focusing on "what is important"), to efficiently extract deep features by alternating between lightweight Transformer layers that employ partial spatial attention and partial channel attention. By alternately using partial spatial attention instantiation followed by partial channel attention instantiation and partial channel attention instantiation followed by partial operator instantiation, deep image features can be efficiently modeled from both "where is important" and "what is important" perspectives. This allows the image enhancement model to learn to both locate key regions and select the key feature types within those regions, resulting in more accurate feature representation.

[0118] Please see Figure 12 The diagram illustrates the alternating processing flow of partial attention. After the image enhancement model completes shallow feature extraction, the obtained sample image features are passed to N lightweight Transformer layers that alternately perform partial spatial attention and partial channel attention to extract attention image features. Specifically, the first lightweight Transformer layer performs deep feature extraction based on partial spatial attention and partial channel attention to obtain intermediate attention image features. These intermediate attention image features are then input into the second lightweight Transformer for deep feature extraction, and so on, until the Nth lightweight Transformer layer completes deep feature extraction to obtain the attention image features. Further, convolutional layers are used to perform feature enhancement processing on these attention image features to obtain the target image features.

[0119] As an example, for the process of deep feature extraction using alternating spatial attention and channel attention, please refer to [link to documentation]. Figure 13The flowchart of partial attention processing is shown as follows: The sample image features are identified as the image features to be extracted. Based on partial spatial attention and partial channel attention, a deep feature extraction process is performed on the image features to be extracted to obtain intermediate attention image features. When it is detected that the number of deep feature extraction processes has not reached the preset processing threshold, the intermediate attention image features are identified as the image features to be extracted, and the process returns to perform a deep feature extraction process based on partial spatial attention and partial channel attention to obtain intermediate attention image features. This process continues until the number of deep feature extraction processes reaches the preset processing threshold. At this point, the intermediate attention image features at which the number of deep feature extraction processes reaches the preset processing threshold are identified as attention image features.

[0120] The following will take the process of extracting intermediate attention image features from a lightweight Transformer layer as an example to explain the main process of deep feature extraction, including: a partial spatial attention block performs deep feature extraction processing on the image features to be extracted based on partial spatial attention to obtain spatial attention image features; further, a partial channel attention block performs deep feature extraction processing on the spatial attention image features based on partial channel attention to obtain intermediate attention image features.

[0121] As an example, such as Figure 14 The diagram shows the structure of a partial spatial attention block. This partial spatial attention block is a processing module based on the Transformer framework, and therefore includes a normalization layer, partial spatial attention, a partial gated network, and residual connections. The image features to be extracted are input into the partial spatial attention block, which then performs feature extraction. The extraction process can be summarized in the following steps.

[0122] (1) Normalize the image features to be extracted to obtain the first spatial normalized features.

[0123] In deep networks, considering the potentially large differences in the scale of outputs from different layers, leading to unstable gradients (too large or too small) during backpropagation, normalization can scale features to a reasonable range, mitigating the gradient problem. Furthermore, inconsistent scales across different feature dimensions can slow down the optimization process (e.g., some dimensions update too quickly, others too slowly); normalization can improve optimization efficiency. For example, a partial spatial attention block can utilize a normalization layer to normalize the image features to be extracted, obtaining the first spatially normalized features. The key technique of this normalization layer is layer normalization: stabilizing the data distribution by standardizing the hidden layer size dimension.

[0124] (2) Based on partial spatial attention, the first spatial normalized features are processed by deep feature extraction to obtain partial spatial attention features.

[0125] To reduce feature redundancy during deep feature extraction in the feature space, such as Figure 15 The diagram shown illustrates the processing of partial spatial attention. Channel segmentation is performed on the first spatially normalized feature to obtain the first part of spatial image features and the second part of spatial image features. The sum of the channel dimensions corresponding to the first part of spatial image features and the channel dimensions corresponding to the second part of spatial image features is the channel dimension corresponding to the first spatially normalized feature.

[0126] Furthermore, deep feature extraction processing, namely spatial window self-attention processing, is performed on the first part of the spatial image features based on partial spatial attention to obtain partial spatial self-attention features. Then, channel stitching processing is performed on the partial spatial self-attention features and the second part of the spatial image features to obtain spatial stitched image features. Finally, feature linear processing is performed on the spatial stitched image features to obtain partial spatial attention features.

[0127] As one approach, spatial window self-attention processing can include: performing linear projection processing based on the first part of the spatial image features to obtain spatial query parameters, spatial key parameters, and spatial value parameters; and performing spatial window segmentation processing on the spatial query parameters, spatial key parameters, and spatial value parameters respectively to obtain window query parameters, window key parameters, and window value parameters corresponding to each spatial window. Further, for each spatial window, spatial self-attention processing is performed based on the spatial query parameters, spatial key parameters, and spatial value parameters to obtain window self-attention features; and the window self-attention features corresponding to each spatial window are cascaded to obtain partial spatial self-attention features. The calculation process of spatial window self-attention processing can be shown in formula (2).

[0128] The input features F corresponding to the instantiation part operator in For the first spatial normalized feature, channel segmentation first spatial normalized feature F in The first part of the spatial image features is obtained. Second part spatial image features Combination Figure 16 The spatial window self-attention diagram shown provides a detailed explanation of formula (2). Using a linear layer W... q W k and W v The first part of the spatial image features F p Perform linear projection processing to obtain spatial query parameters. Space key parameters and spatial value parameters The spatial query parameter O, spatial key parameter K, and spatial value parameter V together constitute the core computational unit of the spatial window self-attention mechanism. The spatial query parameter initiates the request, the spatial key parameter is used for matching and measuring relevance, and the spatial value parameter provides the actual information content. By calculating the similarity between the spatial query parameter and the spatial key parameter, and then weighting and summing the spatial value parameters according to the similarity, the spatial self-attention mechanism can dynamically select and focus on important parts of the input features.

[0129]

[0130] Furthermore, the spatial query parameter Q, spatial key parameter K, and spatial value parameter V are divided into non-overlapping spatial windows (with a resolution of N) using the spatial window segmentation process Partition[·]. w ), This represents the window query parameters corresponding to the d-th spatial window after segmentation. This represents the window key parameter corresponding to the d-th spatial window after partitioning. This represents the window value parameter corresponding to the d-th spatial window after segmentation. Optionally, N w It can take the value 8×32.

[0131] Spatial self-attention processing is performed on each spatial window to obtain the window self-attention feature F. d The Softmax function is used to calculate the probability distribution, where B is the dynamic positional bias matrix, which can be learned by a multilayer perceptron. A multilayer perceptron is a neural network composed of multiple fully connected layers. The input layer receives a relative position encoding vector, and the output is the dynamic positional bias. By introducing dynamic positional bias, the spatial window self-attention mechanism can more accurately and flexibly capture the positional relationships between elements in image features, thereby improving the accuracy of feature extraction. The window self-attention feature F corresponding to each spatial window is... d Perform cascading processing This allows us to obtain partial spatial self-attention features.

[0132] Furthermore, the output feature F corresponding to the instantiation part operator out For some spatial self-attention features F swsa Then, the size is transformed, and then it is concatenated with the second part of the spatial image feature channel and subjected to feature linear processing to obtain partial spatial attention features. The specific calculation process is shown in formula (3).

[0133]

[0134] Among them, F psaFor partial spatial attention features, Reshape(·) is a size transformation used to transform the partial spatial self-attention features F. swsa Feature size transformation back to H×W×C p . This is for channel splicing. W is the projection matrix corresponding to the eigenlinear processing.

[0135] (3) Based on the features of the image to be extracted, perform residual connection processing on some spatial attention features, and normalize the result of the residual connection processing to obtain the second spatial normalized features.

[0136] like Figure 14 The diagram shows the structure of a partial spatial attention block. After the partial spatial attention outputs partial spatial attention features, residual connections can be performed between the image features to be extracted and the partial spatial attention features. Furthermore, the result of the residual connection processing is normalized to obtain the second spatially normalized features. Residual connection processing is mainly used to prevent network degradation.

[0137] (4) Extract context features from the second spatial normalized features to obtain spatial context image features.

[0138] Besides the attention mechanism, the feedforward network is another important component of the Transformer architecture, typically consisting of a non-linear activation layer and two linear projection layers to further propagate information. However, this purely linear layer structure has limited ability to utilize the local context of deep features, which is a crucial reference for high-quality image enhancement. To improve the quality of image enhancement and increase model computational efficiency, embodiments of this disclosure propose replacing the feedforward network in the general Transformer layers with a partially gated network. The partially gated network utilizes more efficient partial convolutions to model the local context. Furthermore, considering that the linear projection layers may introduce redundant channel information, embodiments of this disclosure further incorporate a gating mechanism to help achieve efficient propagation of feature information.

[0139] As an example, the second spatially normalized features are input into a partially gated network. The partially gated network performs partial convolution processing on the second spatially normalized features to obtain partial spatial convolution features. Then, channel segmentation processing is performed on the partial spatial convolution features to obtain first and second partial spatial convolution features. The sum of the channel dimensions corresponding to the first and second partial spatial convolution features is the channel dimension corresponding to the partial spatial convolution features. Further, the partially gated network performs nonlinear processing on the first partial spatial convolution features to obtain partial nonlinear spatial convolution features. Based on the second partial spatial convolution features and the partial nonlinear spatial convolution features, gated modulation processing is performed to obtain spatial context image features. The calculation process for context feature extraction by the partially gated network is shown in formula (4).

[0140]

[0141] Combination Figure 17 The schematic diagram of the partial gated network shown explains formula (4). The partial gated network includes non-convolutional layers, channel splitting, activation functions, and linear layers. Normalized features in the second space are then applied. Perform partial convolution processing PConv(F) to obtain partial spatial convolution features. For partial spatial convolution features Perform average channel splitting processing The first part of the spatial convolution features is obtained. Second part spatial convolution features The first part of the spatial convolution features is processed using an activation function (such as the GELU activation function). Second part spatial convolution features Gating modulation is performed; specifically, the first part of the spatial convolutional features are... The input is fed into the activation function, and the result is combined with the second part of the spatial convolution features. Perform element-wise multiplication and input the product into the linear layer W. pgn The spatial context image features F are calculated. pgn .

[0142] As one approach, partial spatial convolution features can include: performing channel segmentation on the second spatial normalized features to obtain a first part of convolution features and a second part of convolution features, wherein the sum of the channel dimensions corresponding to the first part of convolution features and the channel dimensions corresponding to the second part of convolution features is the channel dimension corresponding to the second spatial normalized features; performing partial convolution on the first part of convolution features to obtain spatial local features; further performing channel concatenation on the spatial local features and the second part of convolution features to obtain spatial local concatenated features; and performing feature linearization on the spatial local concatenated features to obtain partial spatial convolution features. The calculation process of partial spatial convolution features is shown in formula (5).

[0143]

[0144] The input features F corresponding to the instantiation part operator in For the second space normalization feature Channel segmentation and second spatial normalized feature F yield the first part of the convolutional feature. Second part of convolution features Using convolution kernel W c The convolutional layer applies the first part of the convolutional features F p Perform partial convolution processing to obtain spatial local features F conv Furthermore, channel concatenation is performed on the spatial local features and the second part of the convolutional features to obtain the spatial local concatenated features. Furthermore, feature linear processing is performed on the spatial local stitched features to obtain partial spatial convolution features.

[0145] (5) Based on the second spatial normalization features, the spatial context image features are subjected to residual connection processing to obtain spatial attention image features.

[0146] like Figure 14 The schematic diagram of the partial spatial attention block shown illustrates that after obtaining the spatial context image features output by the partial gating network, residual connection processing can be performed on the spatial context image features based on the second spatial normalization features to obtain spatial attention image features. Furthermore, partial channel attention blocks can be used to perform deep feature extraction processing on the spatial attention image features based on partial channel attention to obtain intermediate attention image features.

[0147] As an example, such as Figure 18The diagram shows the structure of a partial channel attention block. This partial channel attention block is a processing module based on the Transformer framework, and therefore includes a normalization layer, partial channel attention, a partial gated network, and residual connections. Spatial attention image features are input into the partial spatial attention block, which then extracts features from the spatial attention image. The extraction process can be summarized in the following steps.

[0148] (1) Normalize the spatial attention image features to obtain the first channel normalized features.

[0149] For example, some spatial attention blocks can utilize a normalization layer to normalize the image features to be extracted, obtaining the first spatially normalized features. The key technology of this normalization layer is layer normalization: stabilizing the data distribution by standardizing the dimension of the hidden layer size.

[0150] (2) Based on the partial channel attention, the normalized features of the first channel are processed by deep feature extraction to obtain the partial channel attention features.

[0151] To reduce feature redundancy during deep feature extraction in feature channels, such as Figure 19 The diagram shown illustrates the partial channel attention process. The normalized features of the first channel are processed by channel segmentation to obtain the first part of the channel image features and the second part of the channel image features. The sum of the channel dimensions corresponding to the first part of the channel image features and the channel dimensions corresponding to the second part of the channel image features is the channel dimension corresponding to the normalized features of the first channel.

[0152] Furthermore, deep feature extraction processing, i.e. channel self-attention processing, is performed on the first part of the channel image features based on partial channel attention to obtain partial channel self-attention features. Then, channel stitching processing is performed based on the partial channel self-attention features and the second part of the channel image features to obtain channel stitched image features. Finally, feature linear processing is performed on the channel stitched image features to obtain partial channel attention features.

[0153] As one implementation, channel self-attention processing may include: performing linear projection processing based on the first part of the channel image features to obtain channel query parameters, channel key parameters, and channel value parameters; performing depthwise separable convolution processing on the channel query parameters, channel key parameters, and channel value parameters to obtain corresponding depth channel query parameters, depth channel key parameters, and depth channel value parameters; further performing size transformation processing on the depth channel query parameters, depth channel key parameters, and depth channel value parameters to obtain corresponding target channel query parameters, target channel key parameters, and target channel value parameters; and performing channel self-attention processing based on the target channel query parameters, target channel key parameters, and target channel value parameters to obtain partial channel self-attention features. The calculation process of channel self-attention processing can be shown in formula (6).

[0154]

[0155] The input features F corresponding to the instantiation part operator in The first channel normalized feature is F, which is used for channel segmentation. in The first part of the channel image features is obtained. Second part of the channel image features Combination Figure 20 The diagram illustrating the channel window self-attention provides a detailed explanation of formula (6). This is achieved by using the first part of the channel image features F... p The parameters are sequentially fed into a linear layer and a depthwise separable convolutional layer to generate depth channel query parameters. Depth channel key parameters and depth channel value parameters Furthermore, the depth channel query parameter Q, depth channel key parameter K, and depth channel value parameter V are respectively subjected to size transformation processing Reshape[Q,K,V] to obtain the corresponding target channel query parameters. Target channel key parameters and target channel value parameters Then, channel self-attention processing is performed based on the target channel query parameters, target channel key parameters, and target channel value parameters to obtain partial channel self-attention features F. cwsa Among them, the target channel query parameters Target channel key parameters and target channel value parameters α is a learnable factor, representing a partial channel self-attention feature.

[0156] Furthermore, the output feature F corresponding to the instantiation part operator out For partial channel self-attention features F cwsaThen, the size is transformed, and then it is concatenated with the feature channels of the second part of the image and subjected to feature linear processing to obtain the partial channel attention features. The specific calculation process is shown in formula (7).

[0157]

[0158] Among them, F pca For partial channel attention features, Reshape(·) is a size transformation used to transform the partial channel self-attention features F. cwsa Feature size transformation back to H×W×C p . This is for channel splicing. W is the projection matrix corresponding to the eigenlinear processing.

[0159] (3) Based on the spatial attention image features, residual connection processing is performed on the attention features of some channels, and the result of the residual connection processing is normalized to obtain the normalized features of the second channel.

[0160] (4) Extract context features from the normalized features of the second channel to obtain the channel context image features.

[0161] (5) Based on the normalized features of the second channel, residual connection processing is performed on the channel context image features to obtain the intermediate attention image features.

[0162] As one implementation method, the calculation principle and calculation process of steps (3) to (5) can be found in the above-mentioned steps (3) to (5) corresponding to some spatial attention blocks, and will not be repeated here.

[0163] In the Transformer layer of the image enhancement model, spatial attention image features can be obtained by performing deep feature extraction processing on the image features to be extracted through partial spatial attention blocks based on partial spatial attention. The spatial window self-attention mechanism of partial spatial attention blocks can separate the extracted features into a set of spatial windows and perform self-attention processing within each window. In this way, by dividing the features into multiple spatial windows and using the self-attention mechanism to calculate the interrelationships between elements within each window, the spatial dependencies between features within the window can be captured, thereby better understanding the local structural information in the image.

[0164] Furthermore, based on partial channel attention, partial channel attention blocks can perform deep feature extraction on spatial attention image features to obtain intermediate attention image features. The channel self-attention mechanism of partial channel attention blocks can better capture the correlation and contextual information of features in different channel dimensions by interacting and adjusting between different channels of the features. Thus, by alternating feature extraction processing with partial spatial attention blocks and partial channel attention blocks, the image enhancement model learns to not only locate key regions in the feature space dimension, but also to select key feature types corresponding to key regions in the feature channel dimension. This results in more accurate feature representation, that is, the extraction of accurate deep image features. In this way, even if the model parameter size of the image enhancement model is reduced, the image enhancement model can still extract accurate image features, so that its performance in performing image enhancement tasks is not affected.

[0165] After performing attention feature extraction processing on the sample image features by repeatedly alternating between partial spatial attention and partial channel attention, attention image features corresponding to the sample image features can be obtained. Further, feature enhancement processing is performed on the attention image features to obtain the target image features, thus completing deep feature extraction. Furthermore, image restoration processing can be performed based on the target image features to obtain an enhanced image of the sample image.

[0166] As an example, such as Figure 12 The method described above can perform residual connection processing on the target image features after deep feature extraction based on the sample image features to obtain connected image features, and then perform image restoration processing based on the connected image features. As one implementation method, sub-pixel convolution processing can be performed on the connected image features to obtain an enhanced image after sample image enhancement.

[0167] Step 130: Determine the target loss value based on the difference between the augmented image corresponding to the sample image and the label image corresponding to the sample image.

[0168] In some embodiments, a target loss function can be determined based on the difference between the augmented image corresponding to the sample image and the label image corresponding to the sample image to calculate the target loss value for supervised training of the image augmentation model. The difference between the augmented image and the label image can be a difference in pixel space, representing the closeness of the augmented image and the label image in terms of pixel values. The target loss function can be a general supervised loss function, including at least L1 loss functions, L2 loss functions, etc., and is not limited here. For example, given an augmented image... and label images The corresponding L1 form objective loss function is

[0169] Step 140: Train the image enhancement model using the target loss value to obtain the trained image enhancement model.

[0170] In some embodiments, the image augmentation model can be iteratively trained using a target loss value until a training termination condition is met, resulting in the trained image augmentation model. The training termination condition can be: the target loss value is less than a preset value, the target loss value is within a threshold range, or the number of training iterations reaches a preset number, etc. Optionally, an optimizer can be used to optimize the target loss function, setting the learning rate, batch size, and epoch based on experimental experience. As an example, iterative training of the model can be implemented using deep learning frameworks such as TensorFlow, PyTorch, and Caffe. The optimizer can use AdamW, and the learning rate can be determined based on experimental and empirical values ​​during the training process.

[0171] Please see Figure 21 The diagram shows the training and testing flowchart. The computer device first determines whether a training or testing task needs to be performed. If a training task is performed, a large number of low-quality images are acquired and preprocessed to generate sample images for training. Further, training parameters are set; for example, the number of Transformer layers in the image augmentation model is set to 19, the total number of channels C is 96, and the number of partially modeled channels (Cp) is 24, etc. The image augmentation model is then optimized end-to-end based on the target loss function and the Adam optimization algorithm, updating the learnable parameters in the model. If a testing task is performed, test images are acquired, the trained image augmentation model is loaded, and the test augmented images are obtained and output through forward computation.

[0172] To verify the effectiveness of the methods proposed in this disclosure, the trained image enhancement model is evaluated on four image processing tasks: image super-resolution, image denoising, image deraining, and low-light image enhancement. For the image super-resolution task, the model is trained on the DIV2K dataset, and the public datasets used for testing include Set5, Set14, BSD100, Urban100, and Manga109. For the image denoising task, the model is trained on four combined datasets: DIV2K, Flickr2K, BSD400, and WED, and the public datasets used for testing include CBSD68, Kodak, McMaster, and Urban100. For the image deraining task, the model is trained on the Rain13K dataset, and the public datasets used for testing include Test100, Test1200, Test2800, Rain100H, and Rain100L. For the low-light enhancement task, experiments are conducted on the LOL-v1 and LOL-v2 datasets. Peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are commonly used to measure performance. Following standard practice, both metrics are calculated in the RGB channels.

[0173] Image super-resolution task. This paper demonstrates the effectiveness of the image enhancement model of this disclosure in image super-resolution. The embodiments of this disclosure extensively compare the image enhancement model with state-of-the-art super-resolution networks. Quantitative results are detailed in [link to documentation]. Figure 22As can be seen, compared to the large-scale super-resolution networks highlighted in gray, this disclosure is competitive with mature CNN methods such as HAN (HolisticAttention Network) and NLSA (Non-Local Sparse Attention), whose computational costs are 44 to 64 times that of this disclosure. Compared to state-of-the-art large-scale Transformer-based models such as SwinIR and ELAN, this disclosure still demonstrates competitiveness, with a model size less than 1 / 8 of the former. Compared to lightweight super-resolution models, this disclosure achieves the best results with higher PSNR / SSIM scores in all benchmarks. Even with lower computational overhead, this disclosure significantly outperforms SwinIR-NG. For better comparison, this disclosure further introduces a more lightweight version, reducing the number of attention blocks (Transformer layers) from 19 to 15, namely this disclosure-light, where the model parameter size of the image augmentation model corresponding to this disclosure-light is smaller than that of the image augmentation model corresponding to this disclosure, and its overhead is comparable to the state-of-the-art SRFormer-light. As can be seen, in this case, this disclosure is still comparable to or even better than SRFormer-light. All these results demonstrate that this disclosure effectively balances performance and model size, setting a new state-of-the-art level for the efficiency of image super-resolution.

[0174] To conduct a comprehensive evaluation, the visual results of different methods were further compared. For example... Figure 23 As shown, the output generated by this disclosure closely resembles the actual ground, accurately preserving structural integrity and fine texture details. In contrast, images generated by comparative algorithms often exhibit significant distortion in structure and detail. These observations highlight the superior expressive power of this disclosure in reconstructing structural content and complex textures.

[0175] Image denoising task. This formulation is compared to an introduced lightweight denoising Transformer baseline, and these methods are evaluated in a blind image denoising setting, i.e., training a single model to handle various noise levels. Figure 24 As shown, this disclosure achieves state-of-the-art performance at almost all noise levels. In particular, the more perceptually relevant SSIM index demonstrates a significant advantage of this disclosure over other methods. Figure 25 The qualitative results further support this finding. Comparative methods show residual noise and artifacts, while this disclosure more effectively recovers structural content and fine details, providing clearer and more accurate image reconstruction results.

[0176] Image deraining task. For example... Figure 26A quantitative comparison of this disclosure with previous rain removal methods is presented. Results show that this disclosure consistently achieves the highest PSNR / SSIM scores across multiple datasets. Compared to HINet (Half Instance Normalization Network), this disclosure offers an average PSNR gain of 0.11 dB while using less than 2% of the parameters required by HINet. On the challenging Test100 dataset, this disclosure demonstrates a more significant advantage, outperforming HINet by 0.47 dB. On all benchmark datasets, this disclosure consistently outperforms the Transformer-based rain removal method IDT (Image De-raining Transformer). Figure 27 The visual results provided further highlight the superiority of this disclosure in achieving better visual quality. It can be seen that the comparative algorithm struggles with residual raindrop artifacts, while this disclosure successfully reconstructs the image with minimal artifacts, providing visually superior results.

[0177] Low-light image enhancement task. Figure 28 Quantitative comparisons of different methods on the LOL-v1 and LOL-v2 datasets are presented. It can be seen that this disclosure achieves best or second-best results in PSNR and SSIM scores. Notably, this disclosure achieves a PSNR 1.33 dB higher than the existing state-of-the-art image restoration method, Restormer, on the LOL-v1 dataset, while using only 4% of its parameters. Furthermore, this disclosure outperforms the significantly larger LLFormer (Low-Light Image Enhancement) on LOL-v1 and outperforms KinD-SKF (Semantic-aware Knowledge-guided Framework) on both datasets, although the SSIM difference on LOL-v2 is only 0.001. It is worth noting that LLFormer and KinD-SKF are specifically tailored for low-light enhancement, while this disclosure is a general-purpose image restoration network applicable to various tasks. Figure 29 A visual comparison is shown. It can be seen that the enhanced image generated by this disclosure is natural and has better global and local contrast.

[0178] This embodiment inputs the acquired sample image into an image enhancement model to perform deep feature extraction processing on the sample image features corresponding to the sample image based on partial spatial attention and partial channel attention, obtaining target image features. Image restoration processing is then performed based on the target image features to obtain an enhanced image with quality greater than the sample image. Specifically, partial spatial attention is used for deep feature extraction processing based on partial image features in the feature space, and partial channel attention is used for deep feature extraction processing based on partial image features in the feature channels. Partial image features are features segmented along the channel dimension of the image features to be extracted, including the sample image features and the image features obtained after each feature extraction process. Therefore, when performing further deep feature extraction processing on the sample image features, spatial attention and channel attention feature processing are applied to a portion of the image features to be extracted to reduce the weight parameters used for feature extraction in the image enhancement model, thereby effectively lightweighting the image enhancement model. Furthermore, a target loss value is determined based on the difference between the enhanced image corresponding to the sample image and the label image corresponding to the sample image, and the image enhancement model is trained using the target loss value to obtain the trained image enhancement model. In this embodiment, during the deep feature extraction process, the image enhancement model learns to perform deep feature extraction based on partial spatial attention and partial channel attention on only a portion of the image features in a certain channel dimension. This not only reduces the weight parameters used by the image enhancement model for feature extraction, making the model lightweight, but also enables the model to locate the key regions for extraction in the feature space dimension and filter out key feature types in the feature channel dimension. Thus, even with a reduced model parameter size, the performance of the model in performing image enhancement tasks is not affected, and the computational efficiency of the computer device is effectively improved.

[0179] Another embodiment of this disclosure provides an image enhancement method. This image enhancement method can be executed by a computer device (e.g., a desktop computer, laptop computer, or other terminal device with certain processing capabilities) after step 140 of the above embodiment, for the model inference stage of the target text elimination model. The computer device must at least have storage, computation, and communication functions. Please refer to... Figure 30 , Figure 30 This diagram illustrates the flow chart of the image enhancement method provided in this embodiment. The following will describe the process in conjunction with... Figure 30 The specific steps of this image enhancement method are explained below:

[0180] Step 210: Obtain the image to be enhanced.

[0181] The image to be enhanced is the input image of the trained image enhancement model. Here, H and W represent the height and width of the image to be enhanced, respectively, and 3 indicates that the image to be enhanced has three color (RGB) channels. The image to be enhanced can come from different tasks. For example, in image restoration tasks, it may be a blurry, low-resolution image to be restored. Another example is a security system image captured by a camera; due to poor lighting conditions (e.g., nighttime, cloudy days) or camera resolution limitations, this image may be blurry, noisy, or have low contrast. Yet another example is remote sensing images acquired in geological exploration; field remote sensing images are often interfered with by atmospheric conditions or terrain shadows, and some small geological phenomena, such as small folds and fault lines, may be blurry in the original image. Finally, in industrial inspection, product images acquired by machine vision systems may be difficult to detect and identify because industrial defects (such as scratches, dents, stains, and cracks) are often small, have low contrast, and complex shapes.

[0182] Step 220: Input the image to be enhanced into the trained image enhancement model, and perform deep feature extraction processing on the image features corresponding to the image to be enhanced based on partial spatial attention and partial channel attention to obtain the target image features.

[0183] In some embodiments, the image to be enhanced is input into an image enhancement model. The image enhancement model can perform shallow feature extraction processing on the image to be enhanced to obtain the features of the image to be enhanced. Then, based on partial spatial attention and partial channel attention, deep feature extraction processing is performed on the features of the image to be enhanced to obtain attention image features. Finally, feature enhancement processing is performed on the attention image features to obtain the target image features. As an example, deep feature extraction processing may include performing deep feature extraction processing on the features of the image to be enhanced based on partial spatial attention and partial channel attention to obtain attention image features, and then performing feature enhancement processing on the attention image features to obtain the target image features.

[0184] For example, in the trained image enhancement model, the process of deep feature extraction for partial spatial attention blocks and partial channel attention blocks can be combined. Figure 31 As shown in formula (8).

[0185]

[0186] Image features The input is fed into a normalization layer of a partial spatial attention block, then passes through partial spatial attention (PSA), and is combined with residual connections to obtain the corresponding output features. Then the features By inputting the data into a normalization layer (LN) and then through a partial gated network (PGN), the output features corresponding to the partial spatial attention blocks can be obtained. Furthermore, the features The input is fed into a normalization layer of a partial channel attention block, then passes through partial channel attention (PCA), and is combined with residual connections to calculate the corresponding output features. Then the features By inputting the data into a normalization layer and passing it through a partial gated network (PGN), the output features corresponding to the attention blocks of some channels can be obtained.

[0187] By alternately using partial spatial attention blocks and partial channel attention blocks, long-range dependencies in the spatial and channel dimensions are captured, respectively. In this way, image information can be efficiently propagated and integrated across two different feature dimensions, enabling comprehensive relational interaction and obtaining more accurate image representations to improve the accuracy and quality of image enhancement.

[0188] Step 230: Using the trained image enhancement model, perform image restoration processing based on the target image features to obtain the enhanced target image after the image to be enhanced.

[0189] In some embodiments, residual connection processing is performed on the target image features after deep feature extraction based on the features of the image to be enhanced to obtain connected image features, and then image restoration processing is performed based on the connected image features to obtain the target enhanced image.

[0190] As one implementation method, subpixel convolution processing can be performed on the connected image features to obtain a pixel rearranged image. Further, image restoration processing is performed based on the pixel rearranged image to obtain an enhanced image after sample image enhancement. For example, in film and television special effects production scenarios, the image enhancement model of this disclosure can also be used to create various special effects, such as scene enhancement, character beautification, and image restoration. By performing image enhancement processing on the original image, more realistic and stunning visual effects can be created, enhancing the artistic appeal of the work. In games, the image enhancement model of this disclosure can also be used to improve the quality of game graphics and enhance the visual experience of the game. For example, enhancing the lighting effects, texture details, and color saturation of game scene graphics makes the game graphics more vivid and realistic. The specific application scenarios of the image enhancement model of this disclosure are not limited here.

[0191] Please see Figure 32 This diagram illustrates a structural block diagram of a training apparatus 300 for an image enhancement model provided in an embodiment of this disclosure. The apparatus may include:

[0192] Sample acquisition module 310 is used to acquire sample images;

[0193] The image processing module 320 is used to input the sample image into the image enhancement model, and perform deep feature extraction processing on the sample image features corresponding to the sample image based on partial spatial attention and partial channel attention to obtain target image features, and perform image restoration processing based on the target image features to obtain the enhanced image after the sample image is enhanced.

[0194] The partial spatial attention is used to perform deep feature extraction processing based on partial image features in the feature space, and the partial channel attention is used to perform deep feature extraction processing based on partial image features in the feature channels. The partial image features are partial features segmented along the channel dimension for the image features to be extracted. The image features to be extracted include sample image features and image features obtained after each feature extraction process.

[0195] The loss determination module 330 is used to determine the target loss value based on the difference between the enhanced image corresponding to the sample image and the label image corresponding to the sample image;

[0196] The model training module 340 is used to train the image enhancement model using the target loss value to obtain the trained image enhancement model.

[0197] In some embodiments, the image processing module 320 may include:

[0198] The shallow feature extraction submodule is used to perform shallow feature extraction processing on the sample image to obtain sample image features;

[0199] The deep feature extraction submodule is used to perform deep feature extraction processing on the sample image features based on partial spatial attention and partial channel attention to obtain attention image features;

[0200] The feature enhancement processing submodule is used to perform feature enhancement processing on the attention image features to obtain target image features.

[0201] In some embodiments, the deep feature extraction submodule may include:

[0202] A feature determination unit is used to determine the sample image features as image features to be extracted.

[0203] The feature extraction unit is used to perform a deep feature extraction process on the image features to be extracted based on partial spatial attention and partial channel attention to obtain intermediate attention image features.

[0204] The deep feature unit is used to determine the intermediate attention image features as image features to be extracted when the number of deep feature extraction processes has not reached a preset processing threshold. It then returns to perform a deep feature extraction process on the image features to be extracted based on partial spatial attention and partial channel attention to obtain intermediate attention image features. This process continues until the number of deep feature extraction processes reaches the preset processing threshold. At this point, the intermediate attention image features at which the number of deep feature extraction processes reaches the preset processing threshold are determined as attention image features.

[0205] In some embodiments, the feature extraction unit may include:

[0206] The spatial processing subunit is used to perform deep feature extraction processing on the image features to be extracted based on partial spatial attention, so as to obtain spatial attention image features;

[0207] The channel processing subunit is used to perform deep feature extraction processing on the spatial attention image features based on partial channel attention to obtain intermediate attention image features.

[0208] In some embodiments, the space processing subunit may include:

[0209] A spatial normalization component is used to normalize the features of the image to be extracted to obtain a first spatial normalized feature.

[0210] A spatial attention component is used to perform deep feature extraction processing on the first spatial normalized features based on partial spatial attention to obtain partial spatial attention features.

[0211] The first spatial residual component is used to perform residual connection processing on the partial spatial attention features based on the features of the image to be extracted, and to normalize the result of the residual connection processing to obtain the second spatial normalized features.

[0212] A spatial context component is used to extract context features from the second spatially normalized features to obtain spatial context image features.

[0213] The second spatial residual component is used to perform residual connection processing on the spatial context image features based on the second spatial normalization features to obtain spatial attention image features.

[0214] In some embodiments, the spatial attention component may include:

[0215] A spatial segmentation sub-component is used to perform channel segmentation processing on the first spatial normalized features to obtain a first part of spatial image features and a second part of spatial image features.

[0216] Wherein, the sum of the channel dimension corresponding to the first part of the spatial image features and the channel dimension corresponding to the second part of the spatial image features is the channel dimension corresponding to the first spatial normalized feature;

[0217] The spatial extraction sub-component is used to perform deep feature extraction processing on the first part of the spatial image features based on partial spatial attention to obtain partial spatial self-attention features;

[0218] A spatial stitching sub-component is used to perform channel stitching processing based on the partial spatial self-attention features and the second part of spatial image features to obtain spatial stitched image features;

[0219] The spatial processing sub-component is used to perform feature linear processing on the spatially stitched image features to obtain partial spatial attention features.

[0220] In some embodiments, the spatial extraction sub-component may be specifically used to: perform linear projection processing based on the first portion of spatial image features to obtain spatial query parameters, spatial key parameters, and spatial value parameters; perform spatial window segmentation processing on the spatial query parameters, spatial key parameters, and spatial value parameters respectively to obtain window query parameters, window key parameters, and window value parameters corresponding to each spatial window; for each spatial window, perform spatial self-attention processing based on the spatial query parameters, spatial key parameters, and spatial value parameters to obtain window self-attention features; and perform cascade processing on the window self-attention features corresponding to each spatial window to obtain partial spatial self-attention features.

[0221] In some embodiments, the spatial context component may include:

[0222] The convolution processing sub-component is used to perform partial convolution processing on the second spatially normalized features to obtain partial spatial convolution features;

[0223] A convolution segmentation sub-component is used to perform channel segmentation processing on the partial spatial convolution features to obtain a first part of spatial convolution features and a second part of spatial convolution features;

[0224] Wherein, the sum of the channel dimension corresponding to the first part of the spatial convolution feature and the channel dimension corresponding to the second part of the spatial convolution feature is the channel dimension corresponding to the partial spatial convolution feature;

[0225] A nonlinear sub-component is used to perform nonlinear processing on the first part of the spatial convolution features to obtain partial nonlinear spatial convolution features.

[0226] A gated sub-component is used to perform gated modulation processing based on the second part of spatial convolutional features and the part of nonlinear spatial convolutional features to obtain spatial context image features.

[0227] In some embodiments, the convolution processing sub-component can be specifically used to perform channel segmentation processing on the second spatially normalized feature to obtain a first part of convolutional feature and a second part of convolutional feature; wherein, the sum of the channel dimensions corresponding to the first part of convolutional feature and the channel dimensions corresponding to the second part of convolutional feature is the channel dimension corresponding to the second spatially normalized feature; performing partial convolution processing on the first part of convolutional feature to obtain spatial local feature; performing channel concatenation processing on the spatial local feature and the second part of convolutional feature to obtain spatial local concatenated feature; and performing feature linear processing on the spatial local concatenated feature to obtain partial spatial convolutional feature.

[0228] In some embodiments, the channel processing subunit may include:

[0229] A channel normalization component is used to normalize the spatial attention image features to obtain the first channel normalized features;

[0230] The channel attention component is used to perform deep feature extraction processing on the normalized features of the first channel based on partial channel attention to obtain partial channel attention features.

[0231] The first residual component of the channel is used to perform residual connection processing on the partial channel attention features based on the spatial attention image features, and to normalize the result of the residual connection processing to obtain the normalized features of the second channel.

[0232] The channel context component is used to extract context features from the normalized features of the second channel to obtain channel context image features;

[0233] The second residual component of the channel is used to perform residual connection processing on the channel context image features based on the normalized features of the second channel to obtain intermediate attention image features.

[0234] In some embodiments, the channel attention component may include:

[0235] The channel segmentation sub-component is used to perform channel segmentation processing on the first channel normalized features to obtain a first part of channel image features and a second part of channel image features.

[0236] Wherein, the sum of the channel dimension corresponding to the first part of the channel image features and the channel dimension corresponding to the second part of the channel image features is the channel dimension corresponding to the first channel normalized feature;

[0237] The channel extraction sub-component is used to perform deep feature extraction processing on the first part of the channel image features based on partial channel attention to obtain partial channel self-attention features;

[0238] The channel stitching sub-component is used to perform channel stitching processing based on the partial channel self-attention features and the second partial channel image features to obtain channel stitched image features;

[0239] The channel processing sub-component is used to perform feature linear processing on the channel stitched image features to obtain partial channel attention features.

[0240] In some embodiments, the channel extraction sub-component may be specifically used to: perform linear projection processing based on the first portion of channel image features to obtain channel query parameters, channel key parameters, and channel value parameters; perform depthwise separable convolution processing on the channel query parameters, channel key parameters, and channel value parameters respectively to obtain corresponding depth channel query parameters, depth channel key parameters, and depth channel value parameters; perform size transformation processing on the depth channel query parameters, depth channel key parameters, and depth channel value parameters respectively to obtain corresponding target channel query parameters, target channel key parameters, and target channel value parameters; and perform channel self-attention processing based on the target channel query parameters, target channel key parameters, and target channel value parameters to obtain partial channel self-attention features.

[0241] In some embodiments, the image processing module 320 may include:

[0242] The residual connection submodule is used to perform residual connection processing on the target image features based on the sample image features to obtain connected image features;

[0243] The image restoration submodule is used to perform subpixel convolution processing on the connected image features to obtain the enhanced image after the sample image is enhanced.

[0244] Please see Figure 33 This diagram illustrates a structural block diagram of an image enhancement apparatus 400 provided in an embodiment of the present disclosure. The apparatus may include:

[0245] Image acquisition module 410 is used to acquire the image to be enhanced;

[0246] The image enhancement module 420 is used to input the image to be enhanced into the trained image enhancement model, and to perform deep feature extraction processing on the image features corresponding to the image to be enhanced based on partial spatial attention and partial channel attention to obtain target image features, and to perform image restoration processing based on the target image features to obtain the enhanced target image; wherein, the trained image enhancement model is obtained according to the training method of the image enhancement model.

[0247] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the described devices, modules, etc., can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0248] In the several embodiments provided in this disclosure, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0249] Furthermore, the functional modules in the various embodiments of this disclosure can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0250] like Figure 34 As shown in the figures, this disclosure also provides a computer device 500, which can be in various forms such as a desktop computer or a laptop computer. The computer device 500 includes a processor 510, a memory 520, a power supply 530, and an input unit 540. The memory 520 stores a computer program, which, when called by the processor 510, can execute the various method steps provided in the above embodiments. Those skilled in the art will understand that the structure of the computer device shown in the figures does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0251] Processor 510 may include one or more processing cores. Processor 510 connects to various parts of the entire battery management system using various interfaces and lines. It executes instructions, programs, instruction sets, or program sets stored in memory 520, calls data stored in memory 520, performs various functions and processes data within the battery management system, and performs various functions and processes data within the computer device, thereby providing overall control of the computer device. Optionally, processor 510 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 510 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 510 and may be implemented separately using a communication chip.

[0252] The memory 520 may include random access memory (RAM) or read-only memory (ROM). The memory 520 can be used to store instructions, programs, instruction sets, or program assemblies. The memory 520 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described above. The data storage area may also store data created during the use of the computer device (such as phonebook and audio / video data). Accordingly, the memory 520 may also include a memory controller to provide the processor 510 with access to the memory 520.

[0253] The power supply 530 can be logically connected to the processor 510 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 530 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0254] The input unit 540 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to object setting and function control.

[0255] Although not shown, the computer device 500 may also include an output unit, a display unit, etc., which will not be described in detail here. Specifically, in the embodiments of this disclosure, the processor 510 of the computer device loads the executable files corresponding to the processes of one or more computer programs into the memory 520 according to the following instructions, and the processor 510 runs the data stored in the memory 520, such as telephone books and audio / video data, thereby implementing the various method steps provided in the foregoing embodiments.

[0256] like Figure 35 As shown, this disclosure also provides a computer-readable storage medium 600, which stores a computer program 610. The computer program 610 can be invoked by a processor of a computer device to execute various method steps provided in this disclosure.

[0257] Computer-readable storage media can be electronic storage devices such as flash memory, electrically erasable programmable read-only memory (EEPROM), hard disks, or ROM. Optionally, the computer-readable storage medium can also be a non-volatile computer-readable storage medium. The computer-readable storage medium 600 has storage space for a computer program that performs any of the method steps in the above embodiments. These computer programs can be read from or written to one or more computer program products. The computer programs can be compressed in an appropriate form.

[0258] According to one aspect of this disclosure, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the various method steps provided in the above embodiments.

[0259] The terms "comprising" and "including," and any variations thereof, in this disclosure and the foregoing drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus. "At least one" means one or more, and "more than" means two or more. "Greater than," "less than," "exceeding," etc., are understood to exclude the number itself. "Above," "below," "within," etc., are understood to include the number itself. The terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined objective, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) may be used to implement one or more modules or units. Furthermore, each module or unit may be part of a larger module or unit that includes the functionality of that module or unit.

[0260] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0261] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0262] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a program product. This computer program product is stored in a storage medium and includes several computer programs to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of this disclosure. It should also be understood that the various implementation methods provided by the embodiments of this disclosure can be arbitrarily combined to achieve different technical effects.

[0263] The above are merely preferred embodiments of this disclosure and are not intended to limit this disclosure in any way. Although this disclosure has been disclosed above with reference to preferred embodiments, it is not intended to limit this disclosure. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this disclosure. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this disclosure without departing from the content of the technical solution of this disclosure shall still fall within the scope of the technical solution of this disclosure.

Claims

1. A training method for an image enhancement model, characterized in that, The method includes: Acquire sample images; The sample image is input into the image enhancement model, and deep feature extraction processing is performed on the sample image features corresponding to the sample image based on partial spatial attention and partial channel attention to obtain target image features. Then, image restoration processing is performed based on the target image features to obtain the enhanced image after the sample image is enhanced. The partial spatial attention is used to perform deep feature extraction processing based on partial image features in the feature space, and the partial channel attention is used to perform deep feature extraction processing based on partial image features in the feature channels. The partial image features are partial features segmented along the channel dimension for the image features to be extracted. The image features to be extracted include sample image features and image features obtained after each feature extraction process. The target loss value is determined based on the difference between the augmented image corresponding to the sample image and the label image corresponding to the sample image; The image enhancement model is trained using the target loss value to obtain the trained image enhancement model.

2. The method according to claim 1, characterized in that, The process of performing deep feature extraction on the sample image features corresponding to the sample image based on partial spatial attention and partial channel attention to obtain target image features includes: The sample image is subjected to shallow feature extraction processing to obtain sample image features; Based on partial spatial attention and partial channel attention, deep feature extraction processing is performed on the sample image features to obtain attention image features; The attention image features are subjected to feature enhancement processing to obtain the target image features.

3. The method according to claim 2, characterized in that, The deep feature extraction process, based on partial spatial attention and partial channel attention, is performed on the sample image features to obtain attention image features, including: The features of the sample image are determined as the image features to be extracted; Based on partial spatial attention and partial channel attention, a deep feature extraction process is performed on the image features to be extracted to obtain intermediate attention image features. When the number of deep feature extraction processes is less than the preset processing threshold, the intermediate attention image features are identified as image features to be extracted. Then, the process returns to perform a deep feature extraction process on the image features to be extracted based on partial spatial attention and partial channel attention to obtain intermediate attention image features. This process continues until the number of deep feature extraction processes reaches the preset processing threshold. At this point, the intermediate attention image features at which the number of deep feature extraction processes reaches the preset processing threshold are identified as attention image features.

4. The method according to claim 3, characterized in that, The process of performing a deep feature extraction on the image features to be extracted based on partial spatial attention and partial channel attention to obtain intermediate attention image features includes: Deep feature extraction processing is performed on the image features to be extracted based on partial spatial attention to obtain spatial attention image features; Deep feature extraction is performed on the spatial attention image features based on partial channel attention to obtain intermediate attention image features.

5. The method according to claim 4, characterized in that, The deep feature extraction process based on partial spatial attention of the image features to be extracted, to obtain spatial attention image features, includes: The image features to be extracted are normalized to obtain the first spatially normalized features; Based on partial spatial attention, the first spatial normalized features are subjected to deep feature extraction processing to obtain partial spatial attention features. Based on the image features to be extracted, residual connection processing is performed on the partial spatial attention features, and the result of the residual connection processing is normalized to obtain the second spatial normalized features. Contextual features are extracted from the second spatial normalized features to obtain spatial context image features; Based on the second spatial normalization feature, residual connection processing is performed on the spatial context image feature to obtain spatial attention image feature.

6. The method according to claim 5, characterized in that, The step of performing deep feature extraction processing on the first spatial normalized features based on partial spatial attention to obtain partial spatial attention features includes: The first spatial normalized features are subjected to channel segmentation to obtain the first part of spatial image features and the second part of spatial image features. Wherein, the sum of the channel dimension corresponding to the first part of the spatial image features and the channel dimension corresponding to the second part of the spatial image features is the channel dimension corresponding to the first spatial normalized feature; Based on partial spatial attention, deep feature extraction processing is performed on the first part of the spatial image features to obtain partial spatial self-attention features; Based on the aforementioned partial spatial self-attention features and the second partial spatial image features, channel stitching processing is performed to obtain spatial stitched image features; The spatial stitched image features are subjected to feature linear processing to obtain partial spatial attention features.

7. The method according to claim 6, characterized in that, The step of performing deep feature extraction processing on the first part of the spatial image features based on partial spatial attention to obtain partial spatial self-attention features includes: Based on the first part of the spatial image features, linear projection processing is performed to obtain spatial query parameters, spatial key parameters, and spatial value parameters; The spatial query parameters, spatial key parameters, and spatial value parameters are respectively subjected to spatial window segmentation processing to obtain the window query parameters, window key parameters, and window value parameters corresponding to each spatial window; For each spatial window, spatial self-attention processing is performed based on spatial query parameters, spatial key parameters, and spatial value parameters to obtain window self-attention features; The window self-attention features corresponding to each spatial window are cascaded to obtain partial spatial self-attention features.

8. The method according to claim 5, characterized in that, The step of extracting contextual features from the second spatially normalized features to obtain spatial contextual image features includes: The second spatially normalized features are partially convolutional to obtain partially spatially convolutional features. The aforementioned spatial convolutional features are subjected to channel segmentation to obtain a first part of spatial convolutional features and a second part of spatial convolutional features; Wherein, the sum of the channel dimension corresponding to the first part of the spatial convolution feature and the channel dimension corresponding to the second part of the spatial convolution feature is the channel dimension corresponding to the partial spatial convolution feature; The first part of the spatial convolution features is subjected to nonlinear processing to obtain partial nonlinear spatial convolution features; Based on the second part of spatial convolution features and the part of nonlinear spatial convolution features, gated modulation processing is performed to obtain spatial context image features.

9. The method according to claim 8, characterized in that, The partial convolution processing of the second spatially normalized features to obtain partial spatial convolution features includes: The second spatially normalized feature is subjected to channel segmentation to obtain the first part of convolutional feature and the second part of convolutional feature; Wherein, the sum of the channel dimension corresponding to the first part of the convolutional features and the channel dimension corresponding to the second part of the convolutional features is the channel dimension corresponding to the second spatial normalization feature; The first part of the convolutional features is partially convolutional to obtain spatial local features; The spatial local features and the second part of the convolutional features are subjected to channel splicing to obtain spatial local spliced ​​features; The spatial local splicing features are subjected to feature linear processing to obtain partial spatial convolution features.

10. The method according to claim 4, characterized in that, The process of performing deep feature extraction on the spatial attention image features based on partial channel attention to obtain intermediate attention image features includes: The spatial attention image features are normalized to obtain the first channel normalized features; Based on partial channel attention, deep feature extraction processing is performed on the normalized features of the first channel to obtain partial channel attention features; Based on the spatial attention image features, residual connection processing is performed on the partial channel attention features, and the result of the residual connection processing is normalized to obtain the second channel normalized features. Contextual features are extracted from the normalized features of the second channel to obtain the channel context image features; Based on the second channel normalized features, residual connection processing is performed on the channel context image features to obtain intermediate attention image features.

11. The method according to claim 10, characterized in that, The step of performing deep feature extraction processing on the normalized features of the first channel based on partial channel attention to obtain partial channel attention features includes: The normalized features of the first channel are subjected to channel segmentation processing to obtain the first part of the channel image features and the second part of the channel image features. Wherein, the sum of the channel dimension corresponding to the first part of the channel image features and the channel dimension corresponding to the second part of the channel image features is the channel dimension corresponding to the first channel normalized feature; Based on partial channel attention, deep feature extraction processing is performed on the image features of the first partial channel to obtain partial channel self-attention features; Channel stitching is performed based on the partial channel self-attention features and the second partial channel image features to obtain channel stitched image features; The features of the stitched image are subjected to feature linear processing to obtain partial channel attention features.

12. The method according to claim 11, characterized in that, The step of performing deep feature extraction processing on the first partial channel image features based on partial channel attention to obtain partial channel self-attention features includes: Linear projection processing is performed based on the first part of the channel image features to obtain channel query parameters, channel key parameters, and channel value parameters; The channel query parameters, the channel key parameters, and the channel value parameters are respectively subjected to depthwise separable convolution processing to obtain the corresponding depth channel query parameters, depth channel key parameters, and depth channel value parameters. The depth channel query parameters, the depth channel key parameters, and the depth channel value parameters are respectively subjected to size transformation processing to obtain the corresponding target channel query parameters, target channel key parameters, and target channel value parameters; Channel self-attention processing is performed based on the target channel query parameters, the target channel key parameters, and the target channel value parameters to obtain partial channel self-attention features.

13. The method according to claim 1, characterized in that, The image restoration process based on the target image features to obtain the enhanced image after enhancing the sample image includes: Based on the sample image features, residual connection processing is performed on the target image features to obtain connected image features; The connected image features are subjected to subpixel convolution processing to obtain the enhanced image after the sample image is enhanced.

14. An image enhancement method, characterized in that, The method includes: Obtain the image to be enhanced; The image to be enhanced is input into the trained image enhancement model. Based on partial spatial attention and partial channel attention, deep feature extraction processing is performed on the image features corresponding to the image to be enhanced to obtain target image features. Based on the target image features, image restoration processing is performed to obtain the target enhanced image after the image to be enhanced is obtained. The trained image enhancement model is obtained by the training method of the image enhancement model according to any one of claims 1 to 13.

15. A training device for an image enhancement model, characterized in that, The device includes: The sample acquisition module is used to acquire sample images. The image processing module is used to input the sample image into the image enhancement model, and to perform deep feature extraction processing on the sample image features corresponding to the sample image based on partial spatial attention and partial channel attention to obtain target image features, and to perform image restoration processing based on the target image features to obtain the enhanced image after the sample image is enhanced. The partial spatial attention is used to perform deep feature extraction processing based on partial image features in the feature space, and the partial channel attention is used to perform deep feature extraction processing based on partial image features in the feature channels. The partial image features are partial features segmented along the channel dimension for the image features to be extracted. The image features to be extracted include sample image features and image features obtained after each feature extraction process. The loss determination module is used to determine the target loss value based on the difference between the enhanced image corresponding to the sample image and the label image corresponding to the sample image; The model training module is used to train the image enhancement model using the target loss value to obtain the trained image enhancement model.

16. An image enhancement device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be enhanced. The image enhancement module is used to input the image to be enhanced into the trained image enhancement model, and to perform deep feature extraction processing on the image features corresponding to the image to be enhanced based on partial spatial attention and partial channel attention to obtain target image features, and to perform image restoration processing based on the target image features to obtain the target enhanced image after the image to be enhanced. The trained image enhancement model is obtained by the training method of the image enhancement model according to any one of claims 1 to 13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the training method for the image enhancement model as described in any one of claims 1 to 13, or the image enhancement method as described in claim 14.

18. A computer device, characterized in that, include: Memory; A processor, wherein a computer program is stored in the memory, and when executed by the processor, the computer program implements the training method of the image enhancement model as described in any one of claims 1 to 13, or the image enhancement method as described in claim 14.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method of the image enhancement model as described in any one of claims 1 to 13, or the image enhancement method as described in claim 14.

Images