Image Halation Removal Method and Device Based on Multi-Frequency Processing Technology

Abstract

The present invention relates to the field of image processing technology, and in particular to a method and device for removing image halo based on multi-frequency processing technology. The method comprises constructing and training a network model for removing image halo based on multi-frequency processing technology. The method comprises the following steps: decomposing an input first image into a low-frequency component containing lighting information and a high-frequency component containing detailed content information; inputting the low-frequency component into a Uformer neural network based on a Transformer network structure to remove halo, thereby obtaining a second image; extracting light source information from the second image to obtain a third image containing only the light source; processing the high-frequency component using a residual network to obtain a fourth image; and fusing the third image with the fourth image to obtain a fifth image without halo. The present invention aims to improve the image quality of nighttime imaging, effectively removing halo artifacts while maintaining the integrity of the original light source and image details.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing technology, and in particular relates to an image halo removal method and device based on multi-frequency processing technology. Background Art

[0002] Vignetting is a common problem in photographic imaging. It can be categorized as diffuse vignetting (scattered vignetting) or reflected vignetting (reflected vignetting). These vignetting effects are caused by light scattering within the lens and reflecting off camera surfaces, respectively. Factors such as fingerprints, dust accumulation, and lens degradation can exacerbate vignetting. These vignetting effects not only obscure critical details but also cause color shifts and uneven brightness, further degrading image quality.

[0003] Hardware-based approaches typically reduce vignetting by improving the optical design and materials of camera lenses. While some progress has been made, they are ineffective at handling multiple types of vignetting and are costly. Deep learning-based approaches utilize neural networks to accurately identify and remove vignetting, but their effectiveness at removing multiple types of vignetting while preserving multiple light sources and restoring image details is less than ideal.

[0004] Therefore, how to more accurately locate and remove halo and improve image visual quality remains a difficult problem in image processing. Summary of the Invention

[0005] In view of this, the present invention aims to provide an image halo removal method based on multi-frequency processing technology. By using multi-frequency technology, the image is decomposed into components of different frequencies to retain the light source and information of the original image to the greatest extent, providing a new processing solution for images that need to remove halo.

[0006] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0007] A method for removing image halo based on multi-frequency processing technology includes constructing and training an image halo removal network model based on multi-frequency processing technology, wherein the construction of the image halo removal network model based on multi-frequency processing technology includes the following steps:

[0008] Decomposing an input first image into a low-frequency component containing illumination information and a high-frequency component containing detail content information;

[0009] Inputting the low-frequency component into a Uformer neural network based on a Transformer network structure to remove the halo, thereby obtaining a second image;

[0010] extracting light source information from the second image to obtain a third image containing only the light source;

[0011] Processing the high-frequency component using a residual network (ResNet) to obtain a fourth image;

[0012] The third image and the fourth image are fused to obtain a fifth image without halo.

[0013] Furthermore, the first image is decomposed into a low-frequency component containing illumination information and a high-frequency component containing detailed content information using a Symlet wavelet function.

[0014] Furthermore, the light source information of the second image is extracted by using a GAN conditional generative adversarial network to obtain a third image of only the light source. The calculation formula is:

[0015]

[0016] in, represents the generator network, represents the parameters of the generator, represents the light-only image generated by the generator; represents the discriminator network, represents the image input to the discriminator, i.e. , represents the parameters of the discriminator, Represents the probability that the discriminator judges that the image is a real image; represents the loss function between the generator and the discriminator, Express expectations, represents the distribution of real images, represents the distribution of the input image vector; Represents the optimal parameters of the generator after training is completed, is the third image processed by the light source extraction module.

[0017] Furthermore, the processing of the high-frequency component by using a residual network to obtain a fourth image includes:

[0018] Performing feature extraction on the high-frequency components to obtain low-level features and high-level features;

[0019] Combining the low-level features with the high-level features;

[0020] The high frequency components are subjected to nonlinear transformation and enhancement processing.

[0021] Furthermore, fusing the third image with the fourth image to obtain a fifth image without halo includes:

[0022] The third image and the fourth image are gradually inverse transformed using Symlet wavelet inverse transform and then gradually fused using multi-scale fusion technology to obtain the fifth image.

[0023] In one embodiment of the present invention, the training of the image halo removal network model based on the multi-frequency processing technology includes the following steps:

[0024] Prepare a halo image dataset and divide it into training set, validation set, and test set in proportion;

[0025] Training the image halo removal network model based on the multi-frequency processing technology using the halo image data in the training set;

[0026] Optimizing the parameters of the image halo removal network model based on the multi-frequency processing technology by using the halo image data in the verification set;

[0027] The halo image data in the test set is input into the optimized image halo removal network model based on multi-frequency processing technology.

[0028] Furthermore, the halo image data set includes a halo image, a light source image and a background image, wherein the background image is a night scene image taken by a camera.

[0029] Furthermore, the training of the image halo removal network model based on the multi-frequency processing technology using the halo image data in the training set includes generating a night image with halo using image editing software from the halo image, light source image and background image in the training set.

[0030] The present invention further provides a computer device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate via the bus, and the machine-readable instructions are executed by the processor to execute the steps of the image halo removal method based on multi-frequency processing technology described in the present invention.

[0031] The present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is run by a processor, the image halo removal method based on the multi-frequency processing technology of the present invention is executed.

[0032] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0033] (1) The image halo removal method based on multi-frequency processing technology created by the present invention utilizes multi-frequency technology to decompose the image into components of different frequencies, which can more accurately locate and remove halo while preserving the light source and information of the original image to the greatest extent.

[0034] (2) The image halo removal method based on multi-frequency processing technology created by the present invention uses a wavelet function to decompose the input image into different frequency components, removes the halo in the low-frequency component image, uses a conditional generative adversarial network to extract the light source, and uses a residual network to enhance the high-frequency component. The light source image and the processed high-frequency component are gradually fused through an inverse wavelet transform and a multi-scale fusion method to obtain a high-quality halo-free image. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0036] Figure 1 A schematic diagram of a process for constructing an image halo removal network model based on multi-frequency processing technology according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of the structure of a network model for image halo removal based on multi-frequency processing technology according to an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of a process for training an image halo removal network model based on multi-frequency processing technology according to an embodiment of the present invention;

[0039] Figure 4 This is an example diagram of a night image with halo (first image) input to the image halo removal network model based on multi-frequency processing technology according to an embodiment of the present invention;

[0040] Figure 5 A high-quality halo-free image (fifth image) processed by the image halo removal method based on the multi-frequency processing technology described in an embodiment of the present invention;

[0041] Figure 6 A schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention. Similar elements in different embodiments use associated similar element numbers. In the following embodiments, many detailed descriptions are intended to enable the present invention to be better understood. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, or methods. In some cases, some operations related to the present invention are not shown or described in the specification. This is to avoid the core part of the present invention being overwhelmed by too much description. For those skilled in the art, it is not necessary to describe these related operations in detail. They can fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0043] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other to form various implementation methods. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various orders in the description and the drawings are only for the purpose of clearly describing a certain embodiment and are not intended to be a required order, unless otherwise specified that a certain order must be followed.

[0044] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0045] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0046] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0047] Example 1

[0048] like Figure 1 As shown, the present invention provides an image halo removal method based on multi-frequency processing technology, including constructing and training an image halo removal network model based on multi-frequency processing technology, wherein the construction of the image halo removal network model based on multi-frequency processing technology includes the steps of:

[0049] S11. Decomposing the first input image into a low-frequency component containing illumination information and a high-frequency component containing detailed content information;

[0050] The first image in this embodiment is a night halo image. After input, the first image is decomposed into two-dimensional wavelet components using the Symlet wavelet function. and high frequency components The low-frequency component mainly reflects the overall brightness and lighting information of the image, while the high-frequency component contains the details and edge information of the image. The decomposition formula of the first image is:

[0051]

[0052] in, is the low-frequency component, corresponding to ; is the low-frequency wavelet basis function; , , They are horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components, which together constitute the high-frequency components ; , , are the high-frequency wavelet basis functions in the horizontal, vertical, and diagonal directions respectively; and are the width and height of the image respectively; and are scale and displacement parameters.

[0053] S12. Inputting the low-frequency component into a Uformer neural network based on a Transformer network structure to remove the halo, thereby obtaining a second image;

[0054] More specifically, the Uformer neural network first extracts features from the low-frequency components, then captures global contextual information through the attention mechanism, and repairs and enhances the low-frequency components based on the contextual information, thereby removing halo artifacts in the low-frequency components and obtaining the second image. , the calculation formula for the second image is:

[0055]

[0056] in, represents the extracted feature map, Represents the part of the neural network used for feature extraction; in the attention mechanism, the neural network will extract the feature map according to the extracted To capture global context information, represents the weighted summation process, 、 、 are query, key, and value matrices respectively, is a scaling factor used to prevent the dot product result from being too large, Used to calculate attention weights; Represents the low-frequency component after removing the halo, Represents the part of the neural network used for repair and enhancement.

[0057] S13 extracts light source information from the second image to obtain a third image of only the light source;

[0058] In this embodiment, a GAN conditional generative adversarial network is used to extract light source information from the second image to obtain a third image containing only the light source.

[0059] S14. Processing the high-frequency component using a residual network to obtain a fourth image;

[0060] The residual network is used to enhance the high-frequency components, making the edges of the image smoother, the details richer, and the overall quality improved. In specific implementation, S14 can be performed simultaneously with S12.

[0061] S15. Fusing the third image and the fourth image to obtain a fifth image without halo.

[0062] In a specific implementation, only the third image and the fourth image of the light source are reconstructed; the third image and the fourth image can be gradually inverse-transformed using the Symlet wavelet inverse transform to obtain a fused image; a multi-scale fusion method is used to further improve the clarity and detail of the image while preserving the integrity of the original light source and image details. The calculation formula used in this embodiment is:

[0063]

[0064] in, Represents the image after multi-scale fusion, which contains Three high frequency components and a low-frequency component; Represents the sigmoid activation function, which is used to combine information of different scales. Indicates image scale The importance weight of Indicated on scale The low-frequency or high-frequency components of the upper fusion; The output high-quality halo-free image is the fifth image. represents the inverse wavelet transform function, represents the low-frequency component after fusion, They represent the horizontal high-frequency component, vertical high-frequency component, and diagonal high-frequency component after multi-scale fusion.

[0065] The image halo removal method based on multi-frequency processing technology provided in this embodiment uses wavelet functions for two-dimensional wavelet decomposition, accurately decomposing the image into low-frequency and high-frequency components. The low-frequency component primarily reflects the overall brightness and lighting information of the image. Targeted processing of the low-frequency component can effectively reduce halo phenomena while maintaining the overall brightness and contrast of the image, thereby obtaining a third image of only the light source. The high-frequency component, on the other hand, contains image details and edge information. By enhancing the high-frequency component, the image edges can be made sharper and the details more distinct, resulting in a fourth image. The third image and the fourth image are gradually fused using a multi-scale fusion method. The multi-scale fusion method fully utilizes image information at different scales, gradually and organically combining the light source image and the high-frequency component, thereby generating a high-quality halo-free prediction result.

[0066] Example 2

[0067] Based on the above embodiment 1, step S13 of the image halo removal method based on multi-frequency processing technology provided in this embodiment specifically includes:

[0068] The GAN conditional generative adversarial network is used to extract the light source information from the second image. When extracting the light source information, the generator in the adversarial network is responsible for generating an image containing only the light source information based on the input image, while the discriminator in the adversarial network is responsible for judging whether the generated image is real. Through continuous adversarial training of the generator and the discriminator, the characteristics of the light source information can be gradually learned, and the light source information in the image can be accurately extracted to generate a third image containing only the light source. , the calculation formula used is:

[0069]

[0070] in, represents the generator network, represents the parameters of the generator, represents the light-only image generated by the generator; represents the discriminator network, represents the image input to the discriminator, i.e. , represents the parameters of the discriminator, Represents the probability that the discriminator judges that the image is a real image; represents the loss function between the generator and the discriminator, Express expectations, represents the distribution of real images, represents the distribution of the input image vector; Represents the optimal parameters of the generator after training is completed, The light source-only image after processing by the light source extraction module.

[0071] This embodiment can more accurately identify and process light source information in an image. By extracting light source information through a GAN conditional generative adversarial network, the distribution and intensity of light sources in an image can be more accurately grasped, thereby providing strong support for subsequent light source adjustment and image enhancement.

[0072] Example 3

[0073] On the basis of the above-mentioned embodiment 1 or 2, step S14 of the image halo removal method based on the multi-frequency processing technology provided in this embodiment specifically includes:

[0074] S141. Extract features of high-frequency components to obtain low-level features and high-level features;

[0075] S142. Combine low-level features with high-level features to retain more image details and edge information;

[0076] S143. Performing nonlinear transformation and enhancement processing on the high-frequency components to further improve image clarity and detail.

[0077] The following formula can be used for calculation in specific implementation:

[0078]

[0079] in, represents the feature map extracted from the high-frequency component, Represents the part of the ResNet network used for feature extraction; and Respectively represent Layer and The feature map of the layer, represents the identity mapping, represents the residual function, Indicates the The weight parameters of the layer; represents the enhanced high-frequency component, Represents the part of the ResNet network used for nonlinear transformation and enhancement.

[0080] Example 4

[0081] like Figure 2 As shown, the image halo removal network model based on multi-frequency processing technology provided by the present invention includes an image decomposition module 51, a low-frequency halo removal module 52, a light source extraction module 53, a high-frequency image enhancement module 54, and an image fusion module 55. The image decomposition module 51 is used to implement step S11 in any of the above-mentioned implementations; the low-frequency halo removal module 52 is used to implement step S12 in any of the above-mentioned implementations; the light source extraction module 53 is used to implement step S13 in any of the above-mentioned implementations; the high-frequency image enhancement module 54 is used to implement step S14 in any of the above-mentioned implementations; and the image fusion module 55 is used to implement step S15 in any of the above-mentioned implementations. These details will not be repeated here.

[0082] Example 5

[0083] On the basis of any of the above embodiments, in the image halo removal method based on multi-frequency processing technology provided in this embodiment, as Figure 3 As shown, the training of the image halo removal network model based on multi-frequency processing technology includes the following steps:

[0084] S21. Prepare a halo image dataset and divide it into training, validation, and test sets in proportion.

[0085] The halo image dataset includes halo images, light source images, and background images, wherein the background image is a night scene image captured by a camera. Preferably, the halo images and corresponding light source images used in the halo image dataset are randomly selected from camera-captured images and software-synthesized images with a probability of 50% each.

[0086] S22. The image halo removal network model is trained based on the multi-frequency processing technology by training the halo image data in the training set;

[0087] The halo images, light source images and background images in the training set are used to generate night images with halo using image editing software.

[0088] S23. Optimize the parameters of the image halo removal network model based on multi-frequency processing technology by verifying the halo image data set;

[0089] S24. Input the halo image data in the test set into the optimized image halo removal network model based on multi-frequency processing technology.

[0090] The halo image data in the test set is input into the trained network model as the data sample image, and the model parameters are fine-tuned to finally achieve the best halo removal effect of the model and save the model parameters.

[0091] Example 6

[0092] Input the image halo removal network model based on multi-frequency processing technology constructed and trained by the present invention as follows: Figure 4 The night image with halo (ie, the first image) is shown in FIG. 1 , and the output image (ie, the fifth image) is shown in FIG. Figure 5 As shown. Figure 4 In the night image with halo shown, there are a lot of scattered halos and reflected halos around the light source. In addition, there are artifacts in the middle of the night image with halo caused by light passing through the lens. Figure 5 ), while retaining the original light and firework light sources, it not only effectively removes scattered light halos, reflected light halos and unnecessary artifacts; but also significantly enhances the details such as smoke in the image, and improves the overall contrast.

[0093] Example 7

[0094] Accordingly, according to an embodiment of the present invention, the present invention also provides a computer device, a readable storage medium, and a computer program product.

[0095] Figure 6 FIG. 1 is a structural diagram of a computer device 12 provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0096] like Figure 6 As shown, computer device 12 is represented in the form of a general-purpose computing device. Computer device 12 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Computer device 12 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present inventions described and / or claimed herein.

[0097] Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16 , a system memory 28 , and a bus 18 that connects various system components, including system memory 28 and processing unit 16 .

[0098] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0099] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0100] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 6 Not shown, often called a "hard drive"). Although Figure 6Not shown, a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0101] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methods of the embodiments described herein.

[0102] The computer device 12 can also communicate with one or more external devices 14 (e.g., a keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication can occur via an input / output (I / O) interface 22. Furthermore, the computer device 12 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via a bus 18. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0103] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28 , such as implementing the image halo removal method based on the multi-frequency processing technology provided by the embodiment of the present invention.

[0104] An embodiment of the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, on which a computer program is stored. When the program is executed by a processor, the image halo removal method based on multi-frequency processing technology provided in all the inventive embodiments of this application is implemented.

[0105] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device.

[0106] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0107] The program code that comprises on the computer-readable medium can be transmitted with any appropriate medium, includes but not limited to wireless, electric wire, optical cable, RF etc., or above-mentioned any suitable combination.Can write the computer program code that is used to carry out the operation of the present invention with one or more programming languages or its combination, described programming language comprises object-oriented programming language such as Java, Smalltalk, C++, also comprises conventional procedural programming language--such as " C " language or similar programming language.Program code can be carried out on user's computer completely, partly on user's computer, carry out as an independent software package, partly on user's computer partly on remote computer, or carry out completely on remote computer or server.In the situation that relates to remote computer, remote computer can comprise local area network (LAN) or wide area network (WAN) to be connected to user's computer by the network of any kind, perhaps, can be connected to external computer (for example, utilize Internet service provider to come to connect by Internet).

[0108] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the above-mentioned image halo removal method based on the multi-frequency processing technology when executed by a processor.

[0109] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0110] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for removing image halo based on multi-frequency processing technology, characterized in that: Constructing and training an image halo removal network model based on multi-frequency processing technology, wherein the construction of the image halo removal network model based on multi-frequency processing technology includes the following steps: Decomposing the input first image into a low-frequency component containing illumination information and a high-frequency component containing detailed content information, including decomposing the first image into the low-frequency component containing illumination information and the high-frequency component containing detailed content information using a Symlet wavelet function; Inputting the low-frequency component into a Uformer neural network based on a Transformer network structure to remove the halo, thereby obtaining a second image; extracting light source information from the second image to obtain a third image containing only the light source; Processing the high-frequency component using a residual network to obtain a fourth image; The third image and the fourth image are fused to obtain a fifth image without halo.

2. The image halo removal method based on multi-frequency processing technology according to claim 1, characterized in that: By using the GAN conditional generative adversarial network to extract the light source information from the second image, a third image of only the light source is obtained, and its calculation formula is: in, represents the generator network, represents the parameters of the generator, represents the light-only image generated by the generator; represents the discriminator network, represents the image input to the discriminator, i.e. , represents the parameters of the discriminator, Represents the probability that the discriminator judges that the image is a real image; represents the loss function between the generator and the discriminator, Express expectations, represents the distribution of real images, represents the distribution of the input image vector; Represents the optimal parameters of the generator after training is completed, is the third image processed by the light source extraction module.

3. The image halo removal method based on multi-frequency processing technology according to claim 1, characterized in that: The using a residual network to process the high frequency component to obtain a fourth image includes: Extracting features of the high-frequency components to obtain low-level features and high-level features; Combining the low-level features with the high-level features; The high frequency components are subjected to nonlinear transformation and enhancement processing.

4. The image halo removal method based on multi-frequency processing technology according to claim 1, characterized in that: The fusing of the third image and the fourth image to obtain a fifth image without halo comprises: The third image and the fourth image are gradually inverse transformed using Symlet wavelet inverse transform and then gradually fused using multi-scale fusion technology to obtain the fifth image.

5. The image halo removal method based on multi-frequency processing technology according to claim 1, characterized in that: The training of the image halo removal network model based on the multi-frequency processing technology includes the following steps: Prepare a halo image dataset and divide it into training set, validation set, and test set in proportion; Training the image halo removal network model based on the multi-frequency processing technology using the halo image data in the training set; Optimizing the parameters of the image halo removal network model based on the multi-frequency processing technology by using the halo image data in the verification set; The halo image data in the test set is input into the optimized image halo removal network model based on multi-frequency processing technology.

6. The image halo removal method based on multi-frequency processing technology according to claim 5, characterized in that: The halo image data set includes a halo image, a light source image and a background image, wherein the background image is a night scene image taken by a camera.

7. The image halo removal method based on multi-frequency processing technology according to claim 6, characterized in that: The training of the image halo removal network model based on the multi-frequency processing technology using the halo image data in the training set includes generating a night image with halo using image editing software from the halo image, light source image and background image in the training set.

8. A computer device, characterized in that: include: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate via the bus. When the processor is running, the machine-readable instructions execute the steps of the image halo removal method based on multi-frequency processing technology as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image halo removal method based on the multi-frequency processing technology according to any one of claims 1 to 7 is executed.

Images