Image halo removing method and device based on multi-frequency processing technology

Abstract

The invention relates to the technical field of image processing, in particular to an image halo removal method and device based on a multi-frequency processing technology, and the method comprises the steps: constructing and training an image halo removal network model based on the multi-frequency processing technology, the construction of the image halo removal network model based on the multi-frequency processing technology comprises the following steps: decomposing an input first image into a low-frequency component containing illumination information and a high-frequency component containing detailed content information; inputting the low-frequency component into a Uform neural network based on a Transform network structure to carry out halo removal so as to obtain a second image; extracting light source information from the second image to obtain a third image with only a light source; processing the high-frequency component by using a residual network to obtain a fourth image; and fusing the third image and the fourth image to obtain a halo-free fifth image. The invention aims to improve the image quality of night imaging, effectively remove halo artifacts, and maintain the integrity of original light sources and image details at the same time.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing technology, and in particular relates to an image halo removal method and device based on multi-frequency processing technology. Background Art

[0002] Halos are a common problem in photographic imaging. Halos are mainly divided into scattered halos and reflected halos, which are caused by the scattering of light within the lens and the reflection of light on the internal surfaces of the camera, respectively. Factors such as fingerprints, dust accumulation and lens degradation will exacerbate the generation of halos. These halos not only block key details, but also cause color shift and uneven brightness, further reducing the visual quality of the image.

[0003] Hardware-based methods usually improve the optical design and materials of camera lenses to reduce halos. Although some progress has been made, they are not effective in dealing with multiple types of halos and are costly. Deep learning-based methods use neural networks to accurately identify and remove halos, but their effects in removing multiple types of halos while retaining multiple light sources and restoring image details are not ideal.

[0004] Therefore, how to more accurately locate and remove halos and improve image visual quality remains a difficult problem in image processing. Summary of the invention

[0005] In view of this, the present invention aims to provide an image halo removal method based on multi-frequency processing technology. The multi-frequency technology is used to decompose the image into components of different frequencies to retain the light source and information of the original image to the greatest extent, providing a new processing solution for images that need to remove halo.

[0006] To achieve the above object, the technical solution created by the present invention is implemented as follows: A method for removing image halo based on multi-frequency processing technology includes constructing and training an image halo removal network model based on multi-frequency processing technology, wherein the construction of the image halo removal network model based on multi-frequency processing technology includes the steps of: Decomposing an input first image into a low-frequency component containing illumination information and a high-frequency component containing detailed content information; Inputting the low-frequency component into a Uformer neural network based on a Transformer network structure to remove the halo, thereby obtaining a second image; Extracting light source information from the second image to obtain a third image containing only the light source; Processing the high frequency component using a residual network (Residual Network, ResNet) to obtain a fourth image; The third image is fused with the fourth image to obtain a fifth image without halo.

[0007] Furthermore, the first image is decomposed into a low-frequency component containing illumination information and a high-frequency component containing detailed content information using a Symlet wavelet function.

[0008] Furthermore, by using the GAN conditional generative adversarial network to extract the light source information from the second image, a third image of only the light source is obtained, and the calculation formula is:

[0009] in, represents the generator network, represents the parameters of the generator, represents the light source-only image generated by the generator; represents the discriminator network, represents the image input to the discriminator, that is , represents the parameters of the discriminator, Represents the probability that the discriminator judges that the image is a real image; represents the loss function between the generator and the discriminator, Express expectations, represents the distribution of real images, represents the distribution of the input image vector; It represents the optimal parameters of the generator after training. is the third image processed by the light source extraction module.

[0010] Further, the using a residual network to process the high frequency component to obtain a fourth image includes: Extracting features of the high-frequency components to obtain low-level features and high-level features; Combining the low-level features with the high-level features; The high frequency component is subjected to nonlinear transformation and enhancement processing.

[0011] Furthermore, the fusing the third image with the fourth image to obtain a fifth image without halo comprises: The third image and the fourth image are gradually inverse transformed using Symlet wavelet inverse transform and then gradually fused using multi-scale fusion technology to obtain the fifth image.

[0012] In one embodiment of the present invention, the training of the image halo removal network model based on multi-frequency processing technology includes the steps of: Prepare the halo image dataset and divide it into training set, validation set and test set in proportion; Training the image halo removal network model based on multi-frequency processing technology through the halo image data in the training set; Optimizing the parameters of the image halo removal network model based on multi-frequency processing technology through the halo image data in the verification set; The halo image data in the test set is input into the optimized image halo removal network model based on multi-frequency processing technology.

[0013] Furthermore, the halo image data set includes a halo image, a light source image and a background image, wherein the background image is a night scene image taken by a camera.

[0014] Furthermore, the training of the image halo removal network model based on the multi-frequency processing technology by the halo image data in the training set includes generating a night image with halo by using image editing software from the halo image, light source image and background image in the training set.

[0015] The present invention further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the image halo removal method based on multi-frequency processing technology described in the present invention.

[0016] The present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the image halo removal method based on the multi-frequency processing technology of the present invention is executed.

[0017] Compared with the prior art, the invention can achieve the following beneficial effects: (1) The image halo removal method based on multi-frequency processing technology created by the present invention utilizes multi-frequency technology to decompose the image into components of different frequencies, which can more accurately locate and remove the halo while retaining the light source and information of the original image to the greatest extent.

[0018] (2) The image halo removal method based on multi-frequency processing technology created by the present invention uses a wavelet function to decompose the input image into different frequency components, removes the halo in the low-frequency component image, uses a conditional generative adversarial network to extract the light source, and a residual network to enhance the high-frequency component. The light source image and the processed high-frequency component are gradually fused through an inverse wavelet transform and a multi-scale fusion method to obtain a high-quality halo-free image. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings: Figure 1 A schematic diagram of a process for constructing an image halo removal network model based on multi-frequency processing technology according to an embodiment of the present invention; Figure 2 A schematic diagram of the structure of the image halo removal network model based on the multi-frequency processing technology described in the embodiment of the present invention; Figure 3 A schematic diagram of a process for training an image halo removal network model based on multi-frequency processing technology according to an embodiment of the present invention; Figure 4 An example diagram of a night image (first image) with halo input to the image halo removal network model based on multi-frequency processing technology described in an embodiment of the present invention; Figure 5 A high-quality halo-free image (fifth image) processed by the image halo removal method based on the multi-frequency processing technology described in the embodiment of the present invention; Figure 6 A schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical scheme and advantages of the invention clearer, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the invention and do not constitute a limitation to the invention. Similar components in different embodiments use associated similar component numbers. In the following embodiments, many detailed descriptions are to enable the invention to be better understood. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other components, materials, and methods. In some cases, some operations related to the invention are not shown or described in the specification, in order to avoid the core part of the invention being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0021] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to form various implementation methods. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a necessary sequence, unless otherwise specified that a certain sequence must be followed.

[0022] In the description of the invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the invention, unless otherwise specified, the meaning of "multiple" is two or more.

[0023] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0024] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0025] Example 1 like Figure 1 As shown, the present invention provides an image halo removal method based on multi-frequency processing technology, including constructing and training an image halo removal network model based on multi-frequency processing technology, wherein the construction of the image halo removal network model based on multi-frequency processing technology includes the steps of: S11. Decomposing the input first image into a low-frequency component containing illumination information and a high-frequency component containing detailed content information; The first image in this embodiment is a night halo image. After input, the first image is decomposed into two-dimensional wavelet components using the Symlet wavelet function. and high frequency components ; The low-frequency component mainly reflects the overall brightness and lighting information of the image, while the high-frequency component contains the details and edge information of the image; The decomposition calculation formula of the first image is:

[0026] in, is the low-frequency component, corresponding to ; is the low-frequency wavelet basis function; , , They are horizontal high-frequency components, vertical high-frequency components, and diagonal high-frequency components, which together constitute the high-frequency components. ; , , They are the high-frequency wavelet basis functions in the horizontal, vertical, and diagonal directions respectively; and are the width and height of the image respectively; and are scale and displacement parameters.

[0027] S12. Inputting the low-frequency component into a Uformer neural network based on a Transformer network structure to remove the halo, thereby obtaining a second image; More specifically, the Uformer neural network first extracts features from the low-frequency components, then captures global context information through the attention mechanism, and repairs and enhances the low-frequency components based on the context information, thereby removing halo artifacts in the low-frequency components and obtaining the second image. , the calculation formula for the second image is:

[0028] in, represents the extracted feature map, Represents the part of the neural network used for feature extraction; in the attention mechanism, the neural network will extract features according to the extracted feature map to capture global context information, represents the weighted summation process, , , are query, key, and value matrices respectively, is a scaling factor used to prevent the dot product result from being too large, Used to calculate attention weights; represents the low-frequency component after removing the halo, Represents the part of the neural network used for inpainting and augmentation.

[0029] S13. Extracting light source information from the second image to obtain a third image of only the light source; In this embodiment, the GAN conditional generative adversarial network is used to extract the light source information from the second image to obtain a third image of only the light source.

[0030] S14. Processing the high frequency component using a residual network to obtain a fourth image; The high frequency components are enhanced by using a residual network to make the edges of the image smoother, the details richer, and the overall quality improved. In specific implementation, S14 can be performed simultaneously with S12.

[0031] S15. Fusing the third image with the fourth image to obtain a fifth image without halo.

[0032] In the specific implementation, the third image and the fourth image of only the light source are reconstructed; the third image and the fourth image can be gradually inversely transformed using the Symlet wavelet inverse transform to obtain a fused image; the image clarity and detail expression are further improved by the multi-scale fusion method, while retaining the integrity of the original light source and image details. The calculation formula used in this embodiment is:

[0033] in, Represents the image after multi-scale fusion, which contains Three high frequency components and a low-frequency component; Represents the sigmoid activation function, which is used to combine information of different scales. Indicates the image scale The importance weight of Indicated in scale The low-frequency or high-frequency components of the upper fusion; represents the output high-quality halo-free image, i.e., the fifth image. represents the inverse wavelet transform function, represents the fused low-frequency component, They respectively represent the horizontal high-frequency component, vertical high-frequency component, and diagonal high-frequency component after multi-scale fusion.

[0034] The image halo removal method based on multi-frequency processing technology provided in this embodiment uses wavelet function to perform two-dimensional wavelet decomposition, which can accurately decompose the image into low-frequency components and high-frequency components. The low-frequency components mainly reflect the overall brightness and lighting information of the image. Targeted processing of low-frequency components can effectively reduce the halo phenomenon while maintaining the overall brightness and contrast of the image, thereby obtaining a third image of only the light source; while the high-frequency components contain the details and edge information of the image. By enhancing the high-frequency components, the edges of the image can be sharper and the details can be clearer, thereby obtaining a fourth image. The third image and the fourth image are gradually fused using a multi-scale fusion method. The multi-scale fusion method makes full use of image information at different scales, and gradually and organically combines the light source image and the high-frequency components, thereby generating a high-quality halo-free prediction result.

[0035] Example 2 On the basis of the above-mentioned embodiment 1, step S13 in the image halo removal method based on multi-frequency processing technology provided in this embodiment specifically includes: The GAN conditional generative adversarial network is used to extract the light source information from the second image. When extracting the light source information, the generator in the adversarial network is responsible for generating an image containing only the light source information based on the input image, while the discriminator in the adversarial network is responsible for judging whether the generated image is real. Through continuous adversarial training of the generator and the discriminator, the characteristics of the light source information can be gradually learned, and the light source information in the image can be accurately extracted to generate a third image containing only the light source. , the calculation formula used is:

[0036] in, represents the generator network, represents the parameters of the generator, represents the light source-only image generated by the generator; represents the discriminator network, represents the image input to the discriminator, that is , represents the parameters of the discriminator, Represents the probability that the discriminator judges that the image is a real image; represents the loss function between the generator and the discriminator, Express expectations, represents the distribution of real images, represents the distribution of the input image vector; It represents the optimal parameters of the generator after training. Light source-only image after being processed by the light source extraction module.

[0037] This embodiment can more accurately identify and process the light source information in the image; by extracting the light source information through the GAN conditional generative adversarial network, the light source distribution and intensity in the image can be more accurately grasped, thereby providing strong support for subsequent light source adjustment and image enhancement.

[0038] Example 3 On the basis of the above-mentioned embodiment 1 or 2, step S14 in the image halo removal method based on multi-frequency processing technology provided in this embodiment specifically includes: S141. Extract features of high-frequency components to obtain low-level features and high-level features; S142. Combining low-level features with high-level features to retain more image details and edge information; S143. Perform nonlinear transformation and enhancement processing on the high frequency components, thereby further improving the clarity and detail of the image; In specific implementation, the following formula can be used for calculation:

[0039] in, represents the feature map extracted from the high-frequency component, Represents the part of the ResNet network used for feature extraction; and Respectively represent Layer and The feature map of the layer, represents the identity mapping, represents the residual function, Indicates The weight parameters of the layer; represents the enhanced high-frequency component, Represents the part of the ResNet network used for nonlinear transformation and enhancement.

[0040] Example 4 like Figure 2 As shown, the image halo removal network model based on multi-frequency processing technology provided by the present invention includes an image decomposition module 51, a low-frequency halo removal module 52, a light source extraction module 53, a high-frequency image enhancement module 54 and an image fusion module 55. Among them, the image decomposition module 51 is used to implement step S11 in any of the above implementations; the low-frequency halo removal module 52 is used to implement step S12 in any of the above implementations; the light source extraction module 53 is used to implement step S13 in any of the above implementations; the high-frequency image enhancement module 54 is used to implement step S14 in any of the above implementations; the image fusion module 55 is used to implement step S15 in any of the above implementations; no further details are given here.

[0041] Example 5 Based on any of the above embodiments, in the image halo removal method based on multi-frequency processing technology provided in this embodiment, Figure 3 As shown, the training of the image halo removal network model based on multi-frequency processing technology includes the following steps: S21. Prepare a halo image dataset and divide it into a training set, a validation set, and a test set in proportion; The halo image dataset includes halo images, light source images and background images, wherein the background image is a night scene image taken by a camera. Preferably, the halo images and corresponding light source images used in the halo image dataset are randomly selected from camera-taken images and software-synthesized images with a probability of 50% each.

[0042] S22. The image halo removal network model based on multi-frequency processing technology is trained by halo image data in the training set; The halo images, light source images and background images in the training set are used to generate night images with halo using image editing software.

[0043] S23. Optimizing the parameters of the image halo removal network model based on multi-frequency processing technology by verifying the halo image data in the set; S24. Input the halo image data in the test set into the optimized image halo removal network model based on multi-frequency processing technology.

[0044] The halo image data in the test set is used as a data sample image and input into the trained network model. The parameters of the model are fine-tuned to ultimately achieve the best halo removal effect of the model and save the model parameters.

[0045] Example 6 Input the image halo removal network model based on multi-frequency processing technology constructed and trained by the present invention as follows: Figure 4 The night image with halo (i.e., the first image) is shown in FIG. 1 , and the output image (i.e., the fifth image) is shown in FIG. Figure 5 As shown. It can be found Figure 4 In the night image with halo shown in the figure, there are a lot of scattered halos and reflected halos around the light source. In addition, there are artifacts in the middle of the night image with halo caused by light passing through the lens. Figure 5 ), while retaining the original light and firework light sources, it not only effectively removes scattered halos, reflected halos and unnecessary artifacts; but also significantly enhances the details such as smoke in the image, and improves the overall contrast.

[0046] Example 7 Accordingly, according to an embodiment of the present invention, the present invention also provides a computer device, a readable storage medium and a computer program product.

[0047] Figure 6 FIG. 1 is a schematic diagram of the structure of a computer device 12 provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0048] like Figure 6As shown, computer device 12 is in the form of a general-purpose computing device. Computer device 12 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Computer devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0049] Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16 , a system memory 28 , and a bus 18 that connects various system components including system memory 28 and processing unit 16 .

[0050] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0051] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0052] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the embodiments of the present invention.

[0053] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0054] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the computer device 12, and / or may communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer device 12, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0055] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the image halo removal method based on the multi-frequency processing technology provided by the embodiment of the present invention.

[0056] An embodiment of the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, on which a computer program is stored, wherein when the program is executed by a processor, the image halo removal method based on multi-frequency processing technology provided in all the inventive embodiments of the present application is implemented.

[0057] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, an apparatus, or a device.

[0058] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0059] The program code included in the computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages ​​such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer by any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect to the Internet).

[0060] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the above-mentioned image halo removal method based on the multi-frequency processing technology when executed by a processor.

[0061] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0062] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for removing image halo based on multi-frequency processing technology, characterized in that: Constructing and training an image halo removal network model based on multi-frequency processing technology, wherein the construction of the image halo removal network model based on multi-frequency processing technology includes the steps of: Decomposing an input first image into a low-frequency component containing illumination information and a high-frequency component containing detailed content information; Inputting the low-frequency component into a Uformer neural network based on a Transformer network structure to remove the halo, thereby obtaining a second image; Extracting light source information from the second image to obtain a third image containing only the light source; Processing the high frequency component using a residual network to obtain a fourth image; The third image is fused with the fourth image to obtain a fifth image without halo.

2. The image halo removal method based on multi-frequency processing technology according to claim 1 is characterized in that: The first image is decomposed into a low-frequency component containing illumination information and a high-frequency component containing detailed content information using a Symlet wavelet function.

3. The image halo removal method based on multi-frequency processing technology according to claim 1, characterized in that: By using the GAN conditional generative adversarial network to extract the light source information from the second image, a third image of only the light source is obtained, and its calculation formula is: in, represents the generator network, represents the parameters of the generator, represents the light source-only image generated by the generator; represents the discriminator network, represents the image input to the discriminator, that is , represents the parameters of the discriminator, Represents the probability that the discriminator judges that the image is a real image; represents the loss function between the generator and the discriminator, Express expectations, represents the distribution of real images, represents the distribution of the input image vector; It represents the optimal parameters of the generator after training. is the third image processed by the light source extraction module.

4. The image halo removal method based on multi-frequency processing technology according to claim 1, characterized in that: The using a residual network to process the high frequency component to obtain a fourth image comprises: Extracting features of the high-frequency components to obtain low-level features and high-level features; Combining the low-level features with the high-level features; The high frequency component is subjected to nonlinear transformation and enhancement processing.

5. The image halo removal method based on multi-frequency processing technology according to claim 1, characterized in that: The step of fusing the third image with the fourth image to obtain a fifth image without halo comprises: The third image and the fourth image are gradually inverse transformed using Symlet wavelet inverse transform and then gradually fused using multi-scale fusion technology to obtain the fifth image.

6. The image halo removal method based on multi-frequency processing technology according to claim 1, characterized in that: The training of the image halo removal network model based on the multi-frequency processing technology comprises the following steps: Prepare the halo image dataset and divide it into training set, validation set and test set in proportion; Training the image halo removal network model based on multi-frequency processing technology through the halo image data in the training set; Optimizing the parameters of the image halo removal network model based on multi-frequency processing technology through the halo image data in the verification set; The halo image data in the test set is input into the optimized image halo removal network model based on multi-frequency processing technology.

7. The image halo removal method based on multi-frequency processing technology according to claim 6, characterized in that: The halo image data set includes a halo image, a light source image and a background image, wherein the background image is a night scene image taken by a camera.

8. The image halo removal method based on multi-frequency processing technology according to claim 7, characterized in that: The training of the image halo removal network model based on multi-frequency processing technology by using the halo image data in the training set includes generating a night image with halo by using image editing software from the halo image, light source image and background image in the training set.

9. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are run by the processor, the steps of the image halo removal method based on multi-frequency processing technology as described in any one of claims 1 to 8 are executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image halo removal method based on multi-frequency processing technology as claimed in any one of claims 1 to 8 is executed.

Images