Extreme overexposure image restoration method, system, medium, program product and equipment

Abstract

The invention belongs to the technical field of computer vision and image processing, and provides an extreme overexposure image restoration method and system, a medium, a program product and equipment. The method comprises the following steps: acquiring an overexposure image, and extracting effective pixel features, semi-effective pixel features and invalid pixel features in the overexposure image; based on a Transform channel attention module, constructing a dependency relationship between semi-effective pixel features in the overexposure image and three pixel channels and a dependency relationship between effective pixel features in the overexposure image and three pixel channels, and adjusting global exposure consistency of the overexposure image to obtain a global exposure correction image; one mask is used for guiding the highlight complementation module to process the global exposure correction image, the last mask is used for guiding the highlight complementation module to output and obtain a final restored image, and a restoration result with high quality and consistent vision can be generated.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and in particular, relates to an extreme overexposed image restoration method, system, medium, program product and device. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Image restoration and enhancement technology has always been a research hotspot in the field of computer vision and image processing, especially the restoration task for overexposed images, whose goal is to restore the details lost in the highlight area due to overexposure. The highlight area in extreme overexposed scenes usually loses detail information completely due to pixel value saturation, and traditional restoration methods face significant challenges in dealing with such problems. Existing technologies are mainly divided into the following research directions: exposure correction and image restoration, but these methods still have many shortcomings in extreme overexposed scenes.

[0004] Exposure correction aims to adjust the overall or local exposure level of an image to improve the visual effect of the image. Existing methods include end-to-end models based on global optimization and learning-driven. Traditional optimization methods balance the brightness of an image by adjusting the histogram distribution or illumination model of the image. Such methods, such as the Retinex-based enhancement model, can improve unevenly exposed images to a certain extent, but cannot restore lost details in highlight overexposed areas. Some methods based on convolutional neural networks (CNNs), such as Deep Photo Enhancer, adjust the exposure level of images in an end-to-end manner. Although these methods perform well in improving the overall quality of low-light or overexposed images, they tend to make global adjustments and are difficult to accurately model highlight areas. The image restoration task focuses on filling in the missing or occluded parts of the image to generate visually reasonable content. However, image restoration technology lacks targeted modeling of specific characteristics of overexposed areas, especially when processing valid pixels and invalid pixels at the same time, it is difficult to coordinate the transition between the two, and the generated results often lack global consistency. The generative adversarial network (GAN) and diffusion model methods work well when dealing with small-scale missing images. However, for large-scale lost overexposed areas, the generated results are prone to inconsistent textures or unreasonable content.

[0005] Therefore, although the existing exposure correction and image restoration methods have solved the problem of overexposed images to a certain extent, they still face the following major challenges for extreme overexposed scenes: (1) Exposure correction methods mainly perform global adjustments on effective pixels and cannot solve the problem of missing details in highlight areas; (2) Image restoration methods lack modeling of highlight area characteristics, resulting in inconsistent textures or insufficient details in the generated results; (3) The transition between highlight areas and backgrounds in existing methods is not natural enough, and the generated restoration content lacks global consistency; (4) The lack of high-quality datasets for extreme overexposed scenes limits the generalization ability of the model and its practical application effect. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides an extreme overexposed image restoration method, system, medium, program product and device, which can generate high-quality and visually consistent restoration results.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A first aspect of the present invention provides a method for repairing an extremely overexposed image.

[0009] In one or more embodiments, a method for repairing an extremely overexposed image is provided, comprising:

[0010] Acquire an overexposed image, and extract effective pixel features, semi-effective pixel features, and invalid pixel features from the overexposed image;

[0011] The channel attention module based on Transformer constructs the dependency relationship between the semi-effective pixel features and effective pixel features in the overexposed image and the three channels of the pixel, adjusts the global exposure consistency of the overexposed image, and obtains the global exposure correction image;

[0012] The global exposure correction image is processed by using at least one mask-guided highlight completion module, and the final restored image is obtained by the output of the last mask-guided highlight completion module; wherein, if there are at least two mask-guided highlight completion modules, the input image of the first mask-guided highlight completion module is the global exposure correction image, and the input images of the remaining mask-guided highlight completion modules are the intermediate restored images output by the previous mask-guided highlight completion module.

[0013] As an implementation method, in each of the mask-guided highlight completion modules, under the guidance of the highlight area mask, the highlight area in the input image is identified and the semi-valid pixel features and invalid pixel features therein are extracted and combined to form the highlight area features; the highlight area features are processed in combination with the fast Fourier convolution module to generate a residual image, which is then added to the input image to obtain the corresponding repaired image.

[0014] As an implementation method, if there are at least two mask-guided highlight completion modules, each intermediate repaired image is processed by a 1×1 convolution layer and a sigmoid activation function in sequence to generate a soft mask and input it into the next mask-guided highlight completion module to form a highlight area mask to identify the highlight area in the input image.

[0015] As an implementation method, the fast Fourier convolution module includes two branches, a local branch and a global branch; in each branch, the highlight area feature is used as the input feature; the local branch and the global branch are processed respectively to obtain local features and global features respectively, and then the local features and the global features are connected to form detail features.

[0016] As an implementation method, before inputting into the local branch and the global branch, the input features are divided into two groups by the channel dimension;

[0017] In the local branch, both sets of features are convolved, and then the reconvolution output features are concatenated to form local features;

[0018] In the global branch, one set of features is convolved, and the other set of features is processed by the spectrum transformer to extract the spectral features. The two sets of processed output features are then connected to form global features.

[0019] As an implementation manner, if there are at least two mask-guided highlight completion modules, in the mask-guided highlight completion modules, intermediate supervision is applied to the intermediate restoration image to optimize the mask quality; the intermediate supervision function is:

[0020]

[0021] Among them, G lr is a real value image downsampled by a set ratio; N is the number of mask-guided highlight completion modules, which is a positive integer greater than or equal to 2; represents the intermediate inpainted image of the ith module.

[0022] A second aspect of the present invention provides an extreme overexposed image restoration system.

[0023] In one or more embodiments, an extreme overexposed image restoration system includes:

[0024] A pixel classification unit, which is used to obtain an overexposed image and extract valid pixel features, semi-valid pixel features and invalid pixel features in the overexposed image;

[0025] The exposure correction unit is used to construct the dependency relationship between the semi-effective pixel features and effective pixel features in the overexposed image and the three channels of the pixel based on the channel attention module of Transformer, adjust the global exposure consistency of the overexposed image, and obtain a global exposure corrected image;

[0026] A highlight reconstruction unit is used to process the global exposure correction image using at least one mask-guided highlight completion module, and obtain the final repaired image by the output of the last mask-guided highlight completion module; wherein, if there are at least two mask-guided highlight completion modules, the input image of the first mask-guided highlight completion module is the global exposure correction image, and the input images of the remaining mask-guided highlight completion modules are the intermediate repaired images output by the previous mask-guided highlight completion module.

[0027] A third aspect of the present invention provides a computer-readable storage medium.

[0028] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the extreme overexposed image restoration method as described above.

[0029] A fourth aspect of the present invention provides a computer program product.

[0030] A computer program product includes a computer program / instruction, which implements the steps in the above-mentioned extreme overexposed image restoration method when executed by a processor.

[0031] A fifth aspect of the present invention provides an electronic device.

[0032] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the extreme overexposure image repair method described above are implemented.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) The present invention includes two stages: exposure correction and highlight reconstruction. In the exposure correction stage, the channel attention module of Transformer is used to construct the dependency relationship between the semi-valid pixel features and the valid pixel features in the overexposed image and the three channels of the pixel, and the global exposure consistency of the overexposed image is adjusted. In the highlight reconstruction stage, the semi-valid pixel features and invalid pixel features in the highlight area are processed to generate real details, which solves the problem that the existing methods are difficult to take into account both global and local restoration.

[0035] (2) The present invention introduces a mask-guided highlight completion module, which accurately locates the highlight area under the guidance of the highlight area mask, and processes the highlight area features in combination with the fast Fourier convolution module, thereby enhancing the global information perception. At the same time, the local context is used to generate content with rich details, so that the repaired highlight area is naturally integrated with the surrounding environment in terms of texture, color and lighting, and the fusion effect of the highlight area and the background is significantly improved.

[0036] (3) The present invention also applies intermediate supervision to the intermediate restoration image, optimizes the mask quality, and ensures that the transition between the generated content and the original image is natural and delicate, thereby improving the restoration effect of the overexposed area. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0038] Figure 1 is a flow chart of an extreme overexposed image restoration method according to an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of two stages of exposure correction and highlight reconstruction in the process of repairing an extremely overexposed image according to an embodiment of the present invention;

[0040] Figure 3 is a schematic diagram of the two stages of exposure correction and highlight reconstruction in the process of repairing an extremely overexposed image according to an embodiment of the present invention;

[0041] Figure 4 This is a comparison of the effects of the extreme overexposed image restoration task according to an embodiment of the present invention;

[0042] Figure 5 This is a first example of comparing the performance differences between the extreme overexposed image restoration method according to an embodiment of the present invention and the existing method in the extreme overexposed image restoration task;

[0043] Figure 6 This is a second example of comparing the performance difference between the extreme overexposed image restoration method according to an embodiment of the present invention and the existing method in the extreme overexposed image restoration task;

[0044] Figure 7 is a schematic diagram of the structure of an extreme overexposed image restoration system according to an embodiment of the present invention;

[0045] Figure 8 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0047] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0048] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0049] Figure 1 FIG. 1 is a flow chart of an extremely overexposed image repair method according to an embodiment of the present invention. Figure 1 The extremely overexposed image repair method in the embodiment shown may include:

[0050] S101: Acquire an overexposed image, and extract valid pixel features, semi-valid pixel features, and invalid pixel features from the overexposed image.

[0051] In the specific implementation process, the valid pixel features, semi-valid pixel features and invalid pixel features in the overexposed image can be extracted through a convolution module or other existing image classification algorithms.

[0052] S102: Based on the channel attention module of Transformer, the dependency relationship between the semi-effective pixel features and the effective pixel features in the overexposed image and the three channels of the pixels is constructed, and the global exposure consistency of the overexposed image is adjusted to obtain a global exposure corrected image.

[0053] Step S102 is the exposure correction stage. Figure 2 and Figure 3 ,The Transformer-based channel attention module achieves the adjustment of the global exposure consistency through feature extraction, encoding, and decoding operations.

[0054] Specifically, for example, given the input feature F in , the query vector Q, key K, and value V are obtained by a 1×1 convolution followed by a depthwise convolution, and then reshaped to the corresponding shape. The channel attention map C can be expressed as: C = Softmax(K×Q / λ), where the operation × represents matrix multiplication and λ is a learnable scaling parameter for numerical stability. The modulated vector The formula can be The output features are obtained by a reshape operation followed by a 1×1 convolutional layer. The exposure correction stage implements three-channel dependency modeling of pixels through a Transformer-based channel attention module and adjusts pixel values ​​using global feature distribution.

[0055] The Transformer-based channel attention module models the dependencies between channels and adjusts the brightness and exposure level of the image, thereby achieving consistency in the overall exposure of the image.

[0056] S103: Use at least one mask-guided highlight completion module to process the global exposure correction image, and obtain the final repaired image by the output of the last mask-guided highlight completion module; wherein, if there are at least two mask-guided highlight completion modules, the input image of the first mask-guided highlight completion module is the global exposure correction image, and the input images of the remaining mask-guided highlight completion modules are the intermediate repaired images output by the previous mask-guided highlight completion module.

[0057] Step S103 is the highlight reconstruction stage, which focuses on repairing the missing details in the image due to overexposure. Due to the conversion in the frequency domain, the fast Fourier convolution enables the network to have a receptive field covering the entire image in the early layers. Specifically, in each of the mask-guided highlight completion modules, under the guidance of the highlight area mask, the highlight area in the input image is identified and the semi-valid pixel features and invalid pixel features therein are extracted to form a highlight area feature; the highlight area features are processed in combination with the fast Fourier convolution module to generate a residual image, which is then added to the input image to obtain the corresponding repaired image.

[0058] The fast Fourier convolution module is used in the highlight reconstruction stage to enhance the network's perception of global information and generate high-quality details in combination with local context.

[0059] If there are at least two mask-guided highlight completion modules, each intermediate repaired image is processed by a 1×1 convolution layer and a sigmoid activation function in turn to generate a soft mask and input it into the next mask-guided highlight completion module to form a highlight area mask to identify the highlight area in the input image.

[0060] Specifically, the fast Fourier convolution module includes two branches, a local branch and a global branch; in each branch, the highlight area feature is used as the input feature; the local branch and the global branch are processed respectively to obtain local features and global features respectively, and then the local features and the global features are connected to form detail features.

[0061] In some specific embodiments, before inputting into the local branch and the global branch, the input features are divided into two groups by the channel dimension;

[0062] In the local branch, both sets of features are convolved, and then the reconvolution output features are concatenated to form local features;

[0063] In the global branch, one set of features is convolved, and the other set of features is processed by the spectrum transformer to extract the spectral features. The two sets of processed output features are then connected to form global features.

[0064] Among them, in the spectrum converter, the spatial features are first converted to the spectrum domain, updated to the spectrum data, and then converted back to the spatial format.

[0065] The combination of global and local features enables the model to balance the needs of global and local features during the restoration process, effectively improving the restoration quality of highlight areas and ensuring the authenticity and consistency of the generated results in terms of texture, color, and lighting.

[0066] If there are at least two mask-guided highlight completion modules, in the mask-guided highlight completion modules, intermediate supervision is applied to the intermediate restoration image to optimize the mask quality; the intermediate supervision function is:

[0067]

[0068] Among them, G lr is a real value image downsampled by a set ratio (e.g., 1 / 8); N is the number of mask-guided highlight completion modules, which is a positive integer greater than or equal to 2; Represents the intermediate restoration image of the i-th module. By introducing the intermediate supervision mechanism, the efficient feature transfer at each stage is ensured, further improving the visual effect and authenticity of the final restoration result.

[0069] This embodiment accurately locates the highlight area under the guidance of the highlight area mask, combines fast Fourier convolution to enhance global perception, and uses local convolution to refine the generated content, so that the repaired highlight area blends naturally with the surrounding environment in terms of texture, color and lighting. The entire repair process is processed in two stages, which not only ensures the uniformity of global exposure, but also restores the detail information of the highlight area, thereby generating a high-quality repaired image.

[0070] The final restored image of this embodiment is verified on the EOR4K dataset, showing the significant effect of the extreme overexposed image restoration method of this embodiment on improving image quality.

[0071] Figure 4The effect comparison of the method of the present invention in the task of extremely overexposed image restoration is shown, from left to right, respectively, the input overexposed image, the intermediate result after the first stage of exposure correction, the generated highlight area mask, and the final restoration result. The first picture is the original overexposed input image, and the brightness is too high, resulting in the complete loss of some details. The second picture is the result after the first stage of exposure correction, which adjusts the exposure levels of effective pixels and semi-effective pixels in the image to make the overall brightness more balanced, but the details of the highlight area are still not fully restored. The third picture shows the highlight area mask generated in the second stage, which automatically marks the areas that need to be repaired through the mask-guided highlight completion module, and the mask-guided highlight completion module focuses on the content generation of the highlight area. The fourth picture is the final restoration result, and it can be seen that the details of the original overexposed area are effectively restored, the texture of the character's face and clothing appear clearer, and the transition with the background lighting and color is natural, showing visual consistency. This shows that the method of the present invention can achieve high-quality restoration of overexposed images under multi-stage collaborative restoration and mask guidance.

[0072] Figure 5 and Figure 6 The performance difference between the method of the present invention and the existing methods in the task of extremely overexposed image restoration is demonstrated. Compared with the problems of blur, color distortion and texture inconsistency shown in highlight area restoration by methods such as FECNet, WaveFill, and HDRTVNet, this method shows excellent detail recovery ability and can generate realistic and delicate restoration content while ensuring the natural transition between the highlight area and the surrounding environment. Whether in people, objects or natural scenes, the present invention can restore the lost texture and color details, and the generated results are closer to the real target image, reflecting higher visual consistency and generalization ability.

[0073] Figure 7 is a schematic diagram of the structure of an extreme overexposed image restoration system in an embodiment of the present invention. Figure 1 The extreme overexposure image repair method corresponds to Figure 7 As shown, the extreme overexposed image restoration system in this embodiment may include:

[0074] A pixel classification unit 701 is used to obtain an overexposed image and extract valid pixel features, semi-valid pixel features and invalid pixel features in the overexposed image;

[0075] The exposure correction unit 702 is used to construct the dependency relationship between the semi-effective pixel features and the effective pixel features in the overexposed image and the three channels of the pixel based on the channel attention module of Transformer, adjust the global exposure consistency of the overexposed image, and obtain a global exposure correction image;

[0076] A highlight reconstruction unit 703 is used to process the global exposure correction image using at least one mask-guided highlight completion module, and obtain the final repaired image by the output of the last mask-guided highlight completion module; wherein, if there are at least two mask-guided highlight completion modules, the input image of the first mask-guided highlight completion module is the global exposure correction image, and the input images of the remaining mask-guided highlight completion modules are the intermediate repaired images output by the previous mask-guided highlight completion module.

[0077] Among them, in each of the mask-guided highlight completion modules, under the guidance of the highlight area mask, the highlight area in the input image is identified and the semi-valid pixel features and invalid pixel features therein are extracted, and combined to form the highlight area features; the highlight area features are processed in combination with the fast Fourier convolution module to generate a residual image, which is then added to the input image to obtain the corresponding repaired image.

[0078] If there are at least two mask-guided highlight completion modules, each intermediate repaired image is processed by a 1×1 convolution layer and a sigmoid activation function in turn to generate a soft mask and input it into the next mask-guided highlight completion module to form a highlight area mask to identify the highlight area in the input image.

[0079] Specifically, the fast Fourier convolution module includes two branches, a local branch and a global branch; in each branch, the highlight area feature is used as the input feature; the local branch and the global branch are processed respectively to obtain local features and global features respectively, and then the local features and the global features are connected to form detail features.

[0080] In some specific embodiments, before inputting into the local branch and the global branch, the input features are divided into two groups by the channel dimension;

[0081] In the local branch, both sets of features are convolved, and then the reconvolution output features are concatenated to form local features;

[0082] In the global branch, one set of features is convolved, and the other set of features is processed by the spectrum transformer to extract the spectral features. The two sets of processed output features are then connected to form global features.

[0083] If there are at least two mask-guided highlight completion modules, in the mask-guided highlight completion modules, intermediate supervision is applied to the intermediate restoration image to optimize the mask quality; the intermediate supervision function is:

[0084]

[0085] Among them, G lris a real value image downsampled by a set ratio; N is the number of mask-guided highlight completion modules, which is a positive integer greater than or equal to 2; represents the intermediate inpainted image of the ith module.

[0086] It should be noted here that Figure 7 The modules in the extreme overexposure image restoration system in Figure 1 The various steps in the extreme overexposure image repair method in correspond to each other, and the specific implementation process is the same, which will not be repeated here.

[0087] Reference Figure 8 , a schematic diagram of an electronic device is given. It should be noted that, Figure 8 The electronic device 800 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0088] like Figure 8 As shown, electronic device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In RAM 803, various programs and data required for system operation are also stored. Central processing unit 801, ROM 802 and RAM 803 are connected to each other via a bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0089] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a local area network (LAN) card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0090] When the central processing unit 801 in the electronic device of this embodiment executes the program, the following is achieved: Figure 1 Steps in the method for repairing an extremely overexposed image are shown.

[0091] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, the computer program including a computer program for executing Figure 1 In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 809, and / or installed from a removable medium 811. When the computer program is executed by the central processing unit 801, various functions defined in the apparatus of the present application are executed.

[0092] in, Figure 1 The computer program instructions corresponding to the method shown may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0093] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for restoring an extremely overexposed image, characterized in that: include: Acquire an overexposed image, and extract effective pixel features, semi-effective pixel features, and invalid pixel features from the overexposed image; The channel attention module based on Transformer constructs the dependency relationship between the semi-effective pixel features and effective pixel features in the overexposed image and the three channels of the pixel, adjusts the global exposure consistency of the overexposed image, and obtains the global exposure correction image; The global exposure correction image is processed by using at least one mask-guided highlight completion module, and the final restored image is obtained by the output of the last mask-guided highlight completion module; wherein, if there are at least two mask-guided highlight completion modules, the input image of the first mask-guided highlight completion module is the global exposure correction image, and the input images of the remaining mask-guided highlight completion modules are the intermediate restored images output by the previous mask-guided highlight completion module.

2. The extremely overexposed image restoration method according to claim 1, characterized in that: In each of the mask-guided highlight completion modules, under the guidance of the highlight area mask, the highlight area in the input image is identified and the semi-valid pixel features and invalid pixel features therein are extracted and combined to form the highlight area features; the highlight area features are processed in combination with the fast Fourier convolution module to generate a residual image, which is then added to the input image to obtain the corresponding repaired image.

3. The extremely overexposed image restoration method according to claim 1 or 2, characterized in that: If there are at least two mask-guided highlight completion modules, each intermediate repaired image is processed by a 1×1 convolution layer and a sigmoid activation function in turn to generate a soft mask and input it into the next mask-guided highlight completion module to form a highlight area mask to identify the highlight area in the input image.

4. The extremely overexposed image restoration method according to claim 2, characterized in that: The fast Fourier convolution module includes two branches: a local branch and a global branch; in each branch, the highlight area feature is used as an input feature; After being processed by the local branch and the global branch respectively, the local features and the global features are obtained correspondingly, and then the local features and the global features are connected to form detail features.

5. The extremely overexposed image restoration method according to claim 4, characterized in that: Before entering the local branch and the global branch, the input features are divided into two groups by the channel dimension; In the local branch, both sets of features are convolved, and then the reconvolution output features are concatenated to form local features; In the global branch, one set of features is convolved, and the other set of features is processed by the spectrum transformer to extract the spectral features. The two sets of processed output features are then connected to form global features.

6. The extremely overexposed image restoration method according to claim 1, characterized in that: If there are at least two mask-guided highlight completion modules, in the mask-guided highlight completion modules, intermediate supervision is applied to the intermediate restoration image to optimize the mask quality; the intermediate supervision function is: Among them, G lr is a real value image downsampled by a set ratio; N is the number of mask-guided highlight completion modules, which is a positive integer greater than or equal to 2; represents the intermediate inpainted image of the ith module.

7. An extreme overexposed image restoration system, characterized in that: include: A pixel classification unit, which is used to obtain an overexposed image and extract valid pixel features, semi-valid pixel features and invalid pixel features in the overexposed image; The exposure correction unit is used to construct the dependency relationship between the semi-effective pixel features and effective pixel features in the overexposed image and the three channels of the pixel based on the channel attention module of Transformer, adjust the global exposure consistency of the overexposed image, and obtain a global exposure corrected image; A highlight reconstruction unit is used to process the global exposure correction image using at least one mask-guided highlight completion module, and obtain the final repaired image by the output of the last mask-guided highlight completion module; wherein, if there are at least two mask-guided highlight completion modules, the input image of the first mask-guided highlight completion module is the global exposure correction image, and the input images of the remaining mask-guided highlight completion modules are the intermediate repaired images output by the previous mask-guided highlight completion module.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the extreme overexposed image repair method as described in any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps in the extreme overexposed image repair method as described in any one of claims 1 to 6 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the extreme overexposed image repair method as described in any one of claims 1-6 are implemented.

Images