Image defogging method, device, equipment and storage medium

Abstract

Embodiments of the present application disclose image defogging method, device, equipment and storage medium, the method comprises: in response to image defogging instruction, obtain corresponding image to be handled;Dark channel non-overlapping in image to be handled is searched layer by layer to obtain atmospheric light estimation value;Based on double constraint strategy, obtain transmittance function in image to be handled and carry out energy minimization and guide filtering, obtain optimized transmittance;Atmospheric light estimation value and optimized transmittance are input to the image physical model corresponding to image to be handled, obtain the defogging image corresponding to image to be handled.The embodiments of the present application can be appropriately corroded bright pixel in image by improved dark channel non-overlapping, and the accuracy of atmospheric light estimation value is improved by using layering search.Moreover, based on double constraint strategy, the optimized transmittance of image to be handled is obtained, which can avoid the generation of block effect, and can also prevent the pixel value overflow of the defogging image, and improve the accuracy of obtaining the defogging image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image defogging method, device, equipment and storage medium. BACKGROUND

[0002] Haze weather is an important reason for causing the image quality of the device with image acquisition function to decline, which greatly hinders the performance of the subsequent automatic identification and other intelligent applications of the visual imaging system. Single image defogging usually does not need additional auxiliary information, and therefore has high application value.

[0003] In the single image defogging algorithm, the dark channel prior is a simple and effective defogging method, which can not only accurately estimate the haze concentration distribution, but also effectively restore the haze image. However, the dark channel prior has certain limitations in the algorithm principle. First, when there are pixels brighter than the atmospheric light in the scene, the dark channel prior will fail, and the defogging image will have pixel value overflow. Second, the dark channel prior estimates the transmission rate by using the local constant assumption, which leads to block effects in the defogging image. Therefore, the above two points greatly limit the performance and universality of the dark channel prior in the algorithm. SUMMARY

[0004] Embodiments of the present application provide an image defogging method, device, equipment and storage medium, aiming to solve the problem that the dark channel prior used in the single image defogging of the prior art will cause pixel value overflow in the defogging image when there are pixels brighter than the atmospheric light in the image scene, and lead to block effects in the defogging image.

[0005] In a first aspect, the embodiments of the present application provide an image defogging method, which includes:

[0006] In response to an image defogging instruction, a to-be-processed image corresponding to the image defogging instruction is acquired;

[0007] The non-overlapping dark channels in the to-be-processed image are searched layer by layer to obtain an atmospheric light estimation value;

[0008] The transmission rate function in the to-be-processed image is acquired based on a preset double-constraint strategy, and energy minimization and guided filtering are performed to obtain an optimized transmission rate;

[0009] The atmospheric light estimation value and the optimized transmission rate are input into an image physical model corresponding to the to-be-processed image to obtain a defogging image corresponding to the to-be-processed image.

[0010] In a second aspect, the embodiments of the present application further provide an image defogging device, which includes:

[0011] An initial image acquisition unit is configured to acquire a to-be-processed image corresponding to the image defogging instruction in response to the image defogging instruction;

[0012] An atmospheric light estimation value acquisition unit is configured to perform hierarchical search on non-overlapping dark channels in the to-be-processed image to obtain an atmospheric light estimation value;

[0013] A transmittance optimization unit is configured to acquire a transmittance function in the to-be-processed image based on a preset double-constraint strategy and perform energy minimization and guided filtering to obtain an optimized transmittance;

[0014] A defogging processing unit is configured to input the atmospheric light estimation value and the optimized transmittance into an image physical model corresponding to the to-be-processed image to obtain a defogged image corresponding to the to-be-processed image.

[0015] In a third aspect, an embodiment of the present application further provides a computer device, which comprises a memory and a processor, the memory has stored thereon a computer program, and the processor implements the method in the first aspect when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, the computer program comprises program instructions, and the program instructions can implement the method in the first aspect when executed by a processor.

[0017] The embodiments of the present application provide an image defogging method, device, equipment and storage medium, wherein the method comprises the following steps: acquiring a to-be-processed image corresponding to an image defogging instruction in response to the image defogging instruction; performing hierarchical search on non-overlapping dark channels in the to-be-processed image to obtain an atmospheric light estimation value; acquiring a transmittance function in the to-be-processed image based on a preset double-constraint strategy and performing energy minimization and guided filtering to obtain an optimized transmittance; and inputting the atmospheric light estimation value and the optimized transmittance into an image physical model corresponding to the to-be-processed image to obtain a defogged image corresponding to the to-be-processed image. The embodiments of the present application can appropriately corrode bright pixels in an image by using improved non-overlapping dark channels, and improve the accuracy of the atmospheric light estimation value by using hierarchical search. Moreover, the embodiments of the present application can avoid the generation of block effect and prevent the pixel value overflow of the defogged image by acquiring the optimized transmittance of the to-be-processed image based on the double-constraint strategy, thereby improving the accuracy of the defogged image. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0019] Figure 1 An application scenario diagram of the image defogging method provided by the embodiment of the present application is shown in the figure.

[0020] Figure 2 A flow diagram of the image defogging method provided by the embodiment of the present application is shown in the figure.

[0021] Figure 3 A sub-flow diagram of the image defogging method provided by the embodiment of the present application is shown in the figure.

[0022] Figure 4 Another sub-flow diagram of the image defogging method provided by the embodiment of the present application is shown in the figure.

[0023] Figure 5a An effect diagram of the image to be processed in the image defogging method provided by the embodiment of the present application is shown in the figure.

[0024] Figure 5b An effect diagram of the defogged image in the image defogging method provided by the embodiment of the present application is shown in the figure.

[0025] Figure 6 A schematic block diagram of the image defogging device provided by the embodiment of the present application is shown in the figure.

[0026] Figure 7 A schematic block diagram of the computer device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0028] It should be understood that, when used in the specification and the appended claims, the terms “comprise” and “include” indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clearly indicated by the context, the singular forms “a”, “an” and “the” are intended to include the plural forms.

[0030] It should be further understood that the term "and / or" used in the description and claims of the application means one or more of the associated listed items as well as all possible combinations of the items and includes these combinations.

[0031] The embodiment of the present application provides an image defogging method, device, equipment and storage medium.

[0032] The image defogging method in the embodiment of the present application is applied to a server or a user terminal, and the server or the user terminal is provided with one or more processors, a memory and one or more application programs, wherein the one or more application programs are stored in the memory and are configured to be executed by the processor to realize the image defogging method.

[0033] As shown in Figure 1 As shown in Figure 1 The scene diagram of the image defogging method of the embodiment of the present application is shown in

[0034] In the embodiment of the present application, the image defogging scene includes a server 100 and a plurality of user terminals 200, wherein the server 100 or the user terminal 200 is integrated with an image defogging device and runs a storage medium corresponding to the image defogging method to execute the steps of the image defogging method.

[0035] It can be understood that Figure 1 The server 100 in the specific application scene of the image defogging method shown in the figure, or the device contained in the server 100, or the user terminal 200, or the device contained in the user terminal 200 does not constitute a limitation on the embodiment of the present application, that is, the number and type of devices contained in the specific application scene of the image defogging method, or the number and type of devices contained in each device do not affect the overall implementation of the technical solution in the embodiment of the present application, and can be regarded as equivalent replacement or derivation of the technical solution claimed in the embodiment of the present application.

[0036] The server 100 or the user terminal 200 in the embodiment of the present application is mainly used for: in response to an image defogging instruction, acquiring a to-be-processed image corresponding to the image defogging instruction; performing hierarchical search on non-overlapping dark channels in the to-be-processed image to obtain an atmospheric light estimation value; obtaining a transmittance function in the to-be-processed image based on a preset double-constraint strategy and performing energy minimization and guided filtering to obtain an optimized transmittance; and inputting the atmospheric light estimation value and the optimized transmittance into an image physical model corresponding to the to-be-processed image to obtain a defogging image corresponding to the to-be-processed image.

[0037] Those skilled in the art can understand that Figure 1 The application environment shown in the figure is only one application scene of the present application scheme and does not constitute a limitation on the application scene of the present application scheme, and other application environments can also include more than oneFigure 1 More or less servers 100 or user terminals 200, or network connection relationship of the servers 100, are shown in the figure, for example Figure 1 Only 1 server 100 is shown in the figure, and it can be understood that the specific application scenario of the image defogging method can also include one or more other servers 100, and the specific application scenario is not limited here; the server 100 or the user terminal 200 can also include a memory.

[0038] Figure 2 The figure is a flowchart of the image defogging method provided by the embodiment of the application. As Figure 2 shown, the method includes the following steps S110-S140.

[0039] S110, in response to an image defogging instruction, acquiring a to-be-processed image corresponding to the image defogging instruction.

[0040] In this embodiment, the server or the user terminal is taken as the execution subject to describe the technical solution. When the user has a defogging processing demand for the to-be-processed picture, the image defogging instruction needs to be triggered. For example, taking the server as the execution subject, when the to-be-processed image is obtained based on the user terminal (such as a smart phone, a smart watch, a camera, and the like, which are devices having a shooting and internet connection function), if the to-be-processed image is selected on the user terminal and it is determined that the to-be-processed image needs to be defogged, the image defogging instruction will be triggered on the user terminal. Then, if the user terminal has no local image defogging processing capability, the to-be-processed image and the image defogging instruction are uploaded to the server, and the server receives and responds to the image defogging instruction, so as to acquire the corresponding to-be-processed image.

[0041] For another example, taking the user terminal as the execution subject, when the to-be-processed image is obtained based on the user terminal, if the to-be-processed image is selected on the user terminal and it is determined that the to-be-processed image needs to be defogged, the image defogging instruction will be triggered on the user terminal. Then, if the user terminal has a local image defogging processing capability, the to-be-processed image and the image defogging instruction do not need to be uploaded to the server, but the to-be-processed image corresponding to the image defogging instruction is acquired locally on the user terminal in response to the image defogging instruction. It can be seen that, whether the server or the user terminal is taken as the execution subject, the to-be-processed image is acquired before the image is defogged, for example, the to-be-processed image acquired is as shown in Figure 5a .

[0042] Specifically, the to-be-processed image can be regarded as a physical model of a foggy image, and the following formula (1) is used to represent it:

[0043] I(x) = J(x)t(x) - A(1-t(x)) (1)

[0044] wherein, in formula (1), I(x) is an observed foggy image (i.e. the image to be processed), x is a pixel coordinate, J(x) is a fog-free image to be processed, t(x) is an atmospheric transmittance, and A is a global atmospheric light.

[0045] S120, performing hierarchical search on non-overlapping dark channels in the image to be processed to obtain an atmospheric light estimation value.

[0046] In this embodiment, in order to avoid the interference of bright objects in the image to be processed, hierarchical search can be performed on all non-overlapping dark channels included in the image to be processed, so as to estimate a more accurate atmospheric light estimation value. Since the hierarchical search is performed after the bright pixels in the image to be processed are properly eroded by using the non-overlapping dark channels, the accuracy of the atmospheric light estimation value can be effectively improved compared with directly performing the atmospheric light value estimation after the image to be processed is preprocessed by using the dark channel.

[0047] In an embodiment, as shown in FIG. 1, step S120 comprises: Figure 3

[0048] S121, obtaining a non-overlapping dark channel based on the image to be processed;

[0049] S122, obtaining a final atmospheric light candidate region by performing a quadtree hierarchical search on the non-overlapping dark channels included in the image to be processed;

[0050] S123, obtaining a color vector having a shortest distance from a preset pixel value in the final atmospheric light candidate region as the atmospheric light estimation value.

[0051] In this embodiment, when the non-overlapping dark channel in the image to be processed is obtained, the difference from the original dark channel in the image to be processed is that the image to be processed is divided into non-overlapping sub-blocks P(i) (wherein, i is in a range of 1 to M, and M is the total number of sub-blocks divided into non-overlapping sub-blocks in the image to be processed), and then the sub-block P(i) is replaced by the smallest pixel value in the current sub-block. Specifically, when the non-overlapping dark channel in the image to be processed is obtained, reference is made to the following formula (2):

[0052]

[0053] wherein, in formula (2), J dark (i) is the non-overlapping dark channel corresponding to the i th non-overlapping sub-block, c is a different color channel, P(i) is the i th non-overlapping sub-block in the image to be processed, J c (y) is a fog-free image in red, green and blue three different color channels.

[0054] In an embodiment, step S121 comprises:​

[0055] The image to be processed is evenly divided into a plurality of image sub-blocks of the same size and without repetition, and the minimum pixel value in each image sub-block is taken as the pixel value of all pixel points in the corresponding image sub-block to obtain the non-overlapping dark channel corresponding to each image sub-block.

[0056] In this embodiment, when the image to be processed is obtained, the image to be processed can be more specifically evenly divided into a plurality of image sub-blocks P(i) of the same size and without repetition. For example, the size of the image to be processed is 1800*900*3 (wherein 3 represents that the image to be processed has R, G and B, and 1800 represents the size of the image to be processed in the horizontal direction, and 900 represents the size of the image to be processed in the vertical direction), and if the picture size is 60*30, the image to be processed can be divided into 30 image sub-blocks P(1)-P(30) of 60*30 and without repetition in the order from left to right and from top to bottom.

[0057] Then, taking the non-overlapping dark channel obtaining process of the image sub-block P(1) of the 30 image sub-blocks P(1)-P(30) as an example, the minimum pixel value in the image sub-block P(1) is first obtained, and then the pixel values of all pixel points in the image sub-block P(1) are replaced with the minimum pixel value, so as to obtain the non-overlapping dark channel of the image sub-block P(1).

[0058] After obtaining the non-overlapping dark channel corresponding to each non-overlapping sub-block in the image to be processed, a quadtree hierarchical search can be performed on all non-overlapping dark channels included in the image to be processed to obtain a final atmospheric light candidate region. The quadtree hierarchical search specifically adopts a quadtree index algorithm, that is, a known range of space is equally divided into four equal subspaces, and this process is recursively performed until the level of the tree reaches a certain depth or the division is stopped when a certain requirement is met. When the known range of space (such as the image to be processed) is converted into a quadtree space, a quadtree hierarchical search can be used to search for a target non-overlapping dark channel that can be regarded as a sky region from a plurality of non-overlapping dark channels, so as to be used as a final atmospheric light candidate region.

[0059] After obtaining the final atmospheric light candidate region, a color vector having the shortest distance from a preset pixel value can be obtained from the final atmospheric light candidate region as the atmospheric light estimate value. For example, if the preset pixel value is set to (255, 255, 255), a color vector having the shortest distance from the pixel value of white in the final atmospheric light candidate region is obtained as the atmospheric light estimate value. It can be seen that the atmospheric light estimate value obtained based on this method is closer to the actual atmospheric light

[0060] S130, obtaining the transmittance function in the to-be-processed image based on the preset double-constraint strategy, and performing energy minimization and guided filtering to obtain an optimized transmittance.

[0061] In the embodiment, if the transmittance is obtained after the dark channel is extracted from the to-be-processed image, the obtained transmittance is not accurate due to the image degradation problem of the dark channel prior. In the present application, a energy function composed of a smooth term and a boundary term is constructed based on the piecewise smooth constraint and the scene radiation constraint to improve the accuracy of the estimated transmittance.

[0062] In an embodiment, as shown in Figure 4 S130 comprises:

[0063] S131, obtaining a piecewise smooth constraint condition and a scene radiation constraint condition in the double-constraint strategy, and constructing an energy function corresponding to the to-be-processed image based on the piecewise smooth constraint condition and the scene radiation constraint condition;

[0064] S132, performing energy minimization on the energy function corresponding to the to-be-processed image as the transmittance function to obtain a rough estimated transmittance;

[0065] S133, performing guided filtering on the rough estimated transmittance to obtain the optimized transmittance.

[0066] In the embodiment, when the energy function corresponding to the to-be-processed image is constructed based on the piecewise smooth constraint condition and the scene radiation constraint condition, the following formula (3) can be referred to:

[0067]

[0068] In formula (3), y∈N(x) represents that pixel y is a neighboring point of pixel x, S(x, y) is a piecewise smooth constraint condition corresponding to the smooth term, and B(x) is a scene radiation constraint condition corresponding to the boundary term.

[0069] If the transmittance map of the to-be-processed image satisfies the piecewise smooth constraint, i.e., neighboring pixels of similar colors have similar transmittances, the smooth term S(x, y) is constructed, and the smooth term S(x, y) can be specifically represented by the following formula (4):

[0070]

[0071] In formula (4), t(x) is the transmittance, I(x) is the foggy image in the to-be-processed image, and pixel y is a neighboring point of pixel x.

[0072] The boundary term B(x) is used to narrow down the solution of the transmittance t(x), and the boundary term B(x) can be specifically expressed by the following formula (5):

[0073]

[0074] In the formula (5), L(x) is a label set including all transmittances t(x), each transmittance is limited by its specific label set, and the label set L(x) is established based on the scene radiance constraint.

[0075] More specifically, the image to be processed can be normalized, and a lower boundary constraint t lb (x) corresponding to the following formula (6) can be obtained in combination with the physical model of the foggy image in formula (1) and the scene radiance constraint condition.

[0076]

[0077] In the formula (6), I c (x) is a foggy image of different color channels, A c is an atmospheric light value of different color channels, t(x) is a transmittance, and 0≤t lb (x)≤t(x)≤1.

[0078] In the foggy image, it cannot be proved which pixels are dark pixels because all pixels that do not satisfy min c I c (x)=0 can be explained as dark pixels covered by fog (where I c (x) also represents a foggy image of different color channels). For this problem, the dark channel prior adopts a local constant assumption, but it causes the initial transmittance map to have a block effect. In the present application, each pixel in the image to be processed is assumed to be a potential dark pixel, thereby avoiding the generation of the block effect. That is, the lower boundary of each pixel in the image to be processed is a candidate of t(x), and the label set L(x) is represented by the following formula (7):

[0079] L(x)={t lb (y)|t lb (y)>t lb (x), y∈Ω(x)} (7)

[0080] In the formula (7), t lb (y) represents the lower boundary of pixel y, and if the lower boundary of pixel y is greater than the lower boundary of pixel x, the lower boundary of pixel y is taken as one of the labels in the label set.

[0081] When the energy function corresponding to the image to be processed is obtained, energy minimization processing can be performed thereon to obtain a rough estimated transmittance.

[0082] In an embodiment, the step S132 comprises:

[0083] The energy function is energy-minimized based on an α-expansion strategy to obtain the rough estimation of the transmittance.

[0084] In the embodiment, the energy function is energy-minimized by the α-expansion strategy, i.e., the α-expansion algorithm. Each time the α-expansion adjusts one label in the label set L(x) to make the energy function decrease. All labels in the label set L(x) are traversed until the energy function cannot decrease in all one expansion adjustments of the label. Then the algorithm stops, and the optimal solution is obtained as the rough estimation of the transmittance.

[0085] Since some redundant noise and details are introduced in the process of calculating the rough estimation of the transmittance, resulting in some visual artifacts in the image to be processed. At this time, the rough estimation of the transmittance can be guided filtered to obtain the optimized transmittance.

[0086] In an embodiment, the step S133 comprises:

[0087] The rough estimation of the transmittance is guided filtered by edge-preserving smoothing processing to obtain the optimized transmittance.

[0088] In the embodiment, when the rough estimation of the transmittance is guided filtered, the edge-preserving smoothing processing can be specifically used for guided filtering, so as to suppress the visual artifacts and obtain the optimized transmittance. The guided filter is a kind of edge-preserving smoothing operator like the bilateral filter, but has better effect near the edge.

[0089] S140, input the atmospheric light estimation value and the optimized transmittance into the image physical model corresponding to the image to be processed to obtain a defogging image corresponding to the image to be processed.

[0090] In the embodiment, since the image physical model corresponding to the image to be processed has been obtained in advance and can be represented by the above formula (1), the defogging image J(x) corresponding to the image to be processed can be represented by the following formula (8):

[0091]

[0092] Wherein, in formula (8) I(x) is the image to be processed, t'(x) is the latest obtained optimized transmittance, and A' is the atmospheric light estimation value adjusted as the global atmospheric light. Based on the above formula (8), the defogging image corresponding to the image to be processed can be obtained, for example, the obtained defogging image is as shown in Figure 5b ​

[0093] In an embodiment, step S140 comprises:

[0094] An image physical model corresponding to the image to be processed is acquired, and the image physical model is adjusted based on a preset lower limit transmittance to obtain an adjusted image physical model.

[0095] The atmospheric light estimation value and the optimized transmittance are input into the adjusted image physical model to obtain a defogged image corresponding to the image to be processed.

[0096] In the present embodiment, in order to avoid the influence of noise and improve the accuracy of obtaining a defogged image, a lower limit transmittance t0 can be introduced into the image physical model corresponding to the image to be processed and adjusted as follows:

[0097]

[0098] The atmospheric light estimation value and the optimized transmittance are input into the adjusted image physical model to obtain a defogged image corresponding to the image to be processed.

[0099] In summary, the present embodiment can not only appropriately erode bright pixels in an image through an improved non-overlapping dark channel, but also improve the accuracy of atmospheric light estimation through hierarchical search. Moreover, the present embodiment can obtain an optimized transmittance of an image to be processed based on a double constraint strategy, which can avoid the generation of blockiness and prevent pixel value overflow in a defogged image, thereby improving the accuracy of obtaining a defogged image.

[0100] Figure 6 is a schematic block diagram of an image defogging device provided by an embodiment of the present application. As shown in Figure 6 Corresponding to the above image defogging method, the present application further provides an image defogging device. The image defogging device comprises units for executing the above image defogging method. Please refer to Figure 6 The image defogging device comprises an initial image acquisition unit 110, an atmospheric light estimation value acquisition unit 120, a transmittance optimization unit 130, and a defogging processing unit 140.

[0101] The initial image acquisition unit 110 is configured to acquire an image to be processed corresponding to an image defogging instruction in response to the image defogging instruction.

[0102] In this embodiment, the server or the user terminal is taken as the execution subject to describe the technical solution. When the user has the haze removal processing demand for the to-be-processed picture, the image haze removal instruction needs to be triggered. For example, taking the server as the execution subject, when the to-be-processed picture is obtained based on the user terminal (such as a smart phone, a smart watch, a camera, and the like, which are devices with the functions of shooting and connecting the Internet), if the to-be-processed picture is selected on the user terminal and it is determined that the to-be-processed picture needs to be processed, the image haze removal instruction is triggered on the user terminal. Then, if the user terminal does not have the image haze removal processing capability locally, the to-be-processed picture and the image haze removal instruction are uploaded to the server, and the corresponding to-be-processed picture is obtained by the server in response to the image haze removal instruction.

[0103] For another example, taking the user terminal as the execution subject, when the to-be-processed picture is obtained based on the user terminal, if the to-be-processed picture is selected on the user terminal and it is determined that the to-be-processed picture needs to be processed, the image haze removal instruction is triggered on the user terminal. Then, if the user terminal has the image haze removal processing capability locally, the to-be-processed picture and the image haze removal instruction do not need to be uploaded to the server, but the corresponding to-be-processed picture is obtained in the user terminal locally in response to the image haze removal instruction. It can be seen that, whether the server or the user terminal is taken as the execution subject, the to-be-processed picture is obtained before the image is processed by haze removal, for example, the to-be-processed picture obtained is as shown in FIG. 1. Figure 5a

[0104] Specifically, the to-be-processed picture can be regarded as a physical model of a hazy image, and the formula (1) is used to represent the to-be-processed picture.

[0105] The atmospheric light estimation value acquisition unit 120 is configured to perform hierarchical search on the non-overlapping dark channels in the to-be-processed picture to obtain an atmospheric light estimation value.

[0106] In this embodiment, in order to avoid the interference of the bright object in the to-be-processed picture, hierarchical search can be performed on all the non-overlapping dark channels included in the to-be-processed picture, so that a more accurate atmospheric light estimation value is estimated. Since the non-overlapping dark channels are used to perform hierarchical search after the bright pixels in the to-be-processed picture are appropriately eroded, compared with directly using the dark channel to pre-process the to-be-processed picture and then estimating the atmospheric light value, the accuracy of the atmospheric light estimation value can be effectively improved.

[0107] In an embodiment, the atmospheric light estimation value acquisition unit 120 is specifically configured to:

[0108] obtain the non-overlapping dark channels based on the to-be-processed picture;

[0109] ​The non-overlapping dark channel of the to-be-processed image is searched by a quadtree hierarchical search to obtain a final atmospheric light candidate region;

[0110] A color vector having a shortest distance from a preset pixel value is obtained in the final atmospheric light candidate region as the atmospheric light estimation value.

[0111] In the embodiment, when the non-overlapping dark channel of the to-be-processed image is obtained, the difference from the original dark channel of the to-be-processed image is that the to-be-processed image is divided into non-overlapping sub-blocks P(i) (wherein i ranges from 1 to M, and M is the total number of sub-blocks divided into non-overlapping sub-blocks in the to-be-processed image), and then the minimum pixel value in the current sub-block is used to replace all pixel values in the sub-block. Specifically, when the non-overlapping dark channel of the to-be-processed image is obtained, reference is made to the above formula (2).

[0112] In an embodiment, the atmospheric light estimation value obtaining unit 120 is further configured to:

[0113] The to-be-processed image is evenly divided into a plurality of image sub-blocks of the same size and without repetition, and the minimum pixel value in each image sub-block is used as the pixel value of all pixel points in the corresponding image sub-block to obtain the non-overlapping dark channel corresponding to each image sub-block.

[0114] In the embodiment, when the non-overlapping dark channel of the to-be-processed image is obtained, the to-be-processed image can be more specifically evenly divided into a plurality of image sub-blocks P(i) of the same size and without repetition. For example, the size of the to-be-processed image is 1800*900*3 (wherein 3 represents that the to-be-processed image has R, G and B, 1800 represents the size of the to-be-processed image in the horizontal direction, and 900 represents the size of the to-be-processed image in the vertical direction), and if the size of the image is 60*30, the to-be-processed image can be divided into 30 image sub-blocks P(1)-P(30) of 60*30 and without repetition in the order from left to right and from top to bottom.

[0115] Then, taking the non-overlapping dark channel of the image sub-block P(1) of the 30 image sub-blocks P(1)-P(30) as an example, the minimum pixel value in the image sub-block P(1) is obtained first, and then all pixel values of the pixel points in the image sub-block P(1) are replaced by the minimum pixel value, so as to obtain the non-overlapping dark channel of the image sub-block P(1).

[0116] When the non-overlapping dark channels corresponding to each non-overlapping sub-block in the to-be-processed image are obtained, a quadtree hierarchical search can be performed on all non-overlapping dark channels included in the to-be-processed image to obtain a final atmospheric light candidate region. The quadtree hierarchical search specifically adopts a quadtree index algorithm, that is, a known range of space is equally divided into four equal subspaces, and this recursive process continues until the level of the tree reaches a certain depth or the division is stopped when a certain requirement is met. When the known range of space (for example, the to-be-processed image can be regarded as a known range of space) is converted into a quadtree space, a target non-overlapping dark channel that can be regarded as a sky region can be searched from the multiple non-overlapping dark channels by using the quadtree hierarchical search, and the target non-overlapping dark channel is used as the final atmospheric light candidate region.

[0117] When the final atmospheric light candidate region is obtained, a color vector having a shortest distance from a preset pixel value can be obtained from the final atmospheric light candidate region as the atmospheric light estimation value. For example, when the preset pixel value is set to (255, 255, 255), a color vector having a shortest distance from a white pixel value in the final atmospheric light candidate region is obtained as the atmospheric light estimation value. It can be seen that the atmospheric light estimation value obtained based on this method is more accurate.

[0118] The transmittance optimization unit 130 is configured to obtain a transmittance function in the to-be-processed image based on a preset double-constraint strategy, perform energy minimization and guided filtering on the transmittance function, and obtain an optimized transmittance.

[0119] In this embodiment, if the transmittance is obtained after the dark channel is extracted from the to-be-processed image, the obtained transmittance is not accurate due to the image degradation problem of the dark channel priori. In this application, a energy function composed of a smoothing term and a boundary term is constructed based on the piecewise smooth constraint and the scene radiation constraint to improve the accuracy of the estimated transmittance.

[0120] In an embodiment, the transmittance optimization unit 130 is specifically configured to:

[0121] obtain a piecewise smooth constraint condition and a scene radiation constraint condition in the double-constraint strategy, and construct an energy function corresponding to the to-be-processed image based on the piecewise smooth constraint condition and the scene radiation constraint condition;

[0122] perform energy minimization on the energy function corresponding to the to-be-processed image as a transmittance function to obtain a roughly estimated transmittance;

[0123] perform guided filtering on the roughly estimated transmittance to obtain the optimized transmittance.

[0124] In the embodiment, when constructing the energy function corresponding to the image to be processed based on the piecewise smooth constraint and the scene radiance constraint, reference can be made to formula (3) as shown above.

[0125] If the transmittance map of the image to be processed satisfies the piecewise smooth constraint, i.e., adjacent pixels of similar color have similar transmittance, a smooth term S(x, y) is constructed, and the smooth term S(x, y) can be specifically represented by formula (4) as shown above.

[0126] The boundary term B(x) is used to narrow the range of the solution of the transmittance t(x), and the boundary term B(x) can be specifically represented by formula (5) as shown above, where L(x) is a label set including all transmittances t(x), each transmittance is limited by its specific label set, and the label set L(x) is established based on the scene radiance constraint.

[0127] More specifically, the image to be processed can be normalized, and a lower boundary constraint t lb (x) corresponding to formula (6) as shown above can be obtained in combination with the physical model of the foggy image in formula (1) and the scene radiance constraint. c c (x) is the foggy image of different color channels, A lb is the atmospheric light value of different color channels, t(x) is the transmittance, and 0≤t c (x)≤t(x)≤1.

[0128] In the foggy image, it cannot be proved which pixels are dark pixels because all pixels not satisfying min c I c (x) = 0 can be explained as dark pixels covered by fog (where I(x) also represents the foggy image of different color channels). For this problem, the dark channel prior adopts the local constant assumption, but this causes the initial transmittance map to have a block effect. In the present application, each pixel in the image to be processed is assumed to be a potential dark pixel, thereby avoiding the generation of the block effect. That is, the lower boundary of each pixel in the image to be processed is a candidate of t(x), and the label set L(x) is represented by formula (7) as shown above, where t lb (y) represents the lower boundary of pixel y, and if the lower boundary of pixel y is greater than the lower boundary of pixel x, the lower boundary of pixel y is taken as one of the labels in the label set.

[0129] After the energy function corresponding to the image to be processed is obtained, energy minimization processing can be performed thereon to obtain a rough estimated transmittance.

[0130] In an embodiment, the energy minimization of the energy function corresponding to the image to be processed as a transmittance function to obtain the rough estimated transmittance comprises:

[0131] The energy minimization of the energy function based on an alpha-expansion strategy to obtain the rough estimated transmittance.

[0132] In the embodiment, the energy minimization of the energy function is specifically performed by an alpha-expansion strategy, that is, an alpha-expansion algorithm. Each time, a label in the label set L(x) is adjusted to make the energy function decrease. All labels in the label set L(x) are traversed until the energy function cannot decrease in all the expansion adjustment labels. Then, the algorithm stops, and thus the optimal solution is obtained as the rough estimated transmittance.

[0133] Since some redundant noise and details are introduced in the process of calculating the rough estimated transmittance, the image to be processed has some visual artifacts. At this time, the rough estimated transmittance can be guided filtered to obtain the optimized transmittance.

[0134] In an embodiment, the guided filtering of the rough estimated transmittance to obtain the optimized transmittance comprises:

[0135] The rough estimated transmittance is guided filtered by edge-preserving smoothing processing to obtain the optimized transmittance.

[0136] In the embodiment, when the rough estimated transmittance is guided filtered, the edge-preserving smoothing processing can be specifically used for guided filtering, so as to suppress the visual artifacts and obtain the optimized transmittance. The guided filtering specifically uses a guided filter. The guided filter is an edge-preserving smoothing operator like a bilateral filter, but has better effect near the edge.

[0137] The defogging processing unit 140 is configured to input the atmospheric light estimation value and the optimized transmittance into an image physical model corresponding to the image to be processed, to obtain a defogging image corresponding to the image to be processed.

[0138] In the embodiment, since the image physical model corresponding to the image to be processed has been obtained in advance and can be represented by the formula (1), the defogging image J(x) corresponding to the image to be processed can be represented by the formula (8). In the formula (8), I(x) is the image to be processed, t'(x) is the latest obtained optimized transmittance, and A' is the atmospheric light estimation value adjusted to be the global atmospheric light. Based on the formula (8), the defogging image corresponding to the image to be processed can be obtained. For example, the obtained defogging image is as shown in FIG. 8. Figure 5b

[0139] ​In an embodiment, the haze removal processing unit 140 is specifically configured to:

[0140] obtain an image physical model corresponding to the image to be processed, and adjust the image physical model based on a preset lower limit transmittance to obtain an adjusted image physical model;

[0141] input the atmospheric light estimation value and the optimized transmittance into the adjusted image physical model to obtain a haze-removed image corresponding to the image to be processed.

[0142] In the embodiment, in order to avoid the influence of noise and improve the accuracy of obtaining the haze-removed image, a lower limit transmittance t0 can be introduced into the image physical model corresponding to the image to be processed, which is adjusted as formula (9) above, and then the atmospheric light estimation value and the optimized transmittance are input into the adjusted image physical model for calculation, so that the haze-removed image without the influence of noise can be obtained.

[0143] In summary, the embodiment can not only appropriately erode the bright pixels in the image through the improved non-overlapping dark channel, but also improve the accuracy of the atmospheric light estimation value through the hierarchical search. Moreover, the optimized transmittance of the image to be processed can be obtained based on the double constraint strategy, which can avoid the generation of the blocking effect and prevent the pixel value overflow of the image after haze removal, thereby improving the accuracy of obtaining the haze-removed image.

[0144] It should be noted that the specific implementation process of the image haze removal device and each unit can be clearly understood by those skilled in the art, and the corresponding description in the foregoing method embodiments can be referred to. For the convenience and brevity of description, the details are not described herein.

[0145] The image haze removal device described above can be implemented in the form of a computer program, which can run on a computer device as shown in Figure 7 .

[0146] Please refer to Figure 7 , Figure 7 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device integrates any image haze removal device provided by an embodiment of the present application.

[0147] Please refer to Figure 7 , the computer device includes a processor 402, a memory and a network interface 405 connected through a system bus 401, wherein the memory can include a storage medium 403 and an internal memory 404.

[0148] The storage medium 403 can store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions, which, when executed, can cause the processor 402 to perform an image haze removal method.

[0149] The processor 402 is configured to provide computing and control capabilities to support the operation of the entire computer device.

[0150] The internal memory 404 provides an environment for the operation of the computer program 4032 in the storage medium 403, and the computer program 4032, when executed by the processor 402, can make the processor 402 execute the image defogging method described above.

[0151] The network interface 405 is configured to perform network communication with other devices. Those skilled in the art can understand that the network interface 405 can be implemented by a network card, a network adapter, a network controller, or the like. Figure 7 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0152] The processor 402 is configured to execute the computer program 4032 stored in the memory to implement the following steps:

[0153] In response to an image defogging instruction, a to-be-processed image corresponding to the image defogging instruction is acquired;

[0154] The non-overlapping dark channels in the to-be-processed image are searched layer by layer to obtain an atmospheric light estimation value;

[0155] The transmittance function in the to-be-processed image is acquired based on a preset double-constraint strategy and energy minimization and guided filtering are performed to obtain an optimized transmittance;

[0156] The atmospheric light estimation value and the optimized transmittance are input into an image physical model corresponding to the to-be-processed image to obtain a defogging image corresponding to the to-be-processed image.

[0157] It should be understood that, in the embodiments of the present application, the processor 402 can be a central processing unit (CPU), and the processor 402 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0158] Those skilled in the art can understand that all or part of the processes in the method of the above-mentioned embodiments can be completed by instructing the relevant hardware by a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the computer system to realize the process steps of the above-mentioned embodiment of the method.

[0159] Therefore, the present application also provides a storage medium. The storage medium can be a computer readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. The program instructions are executed by a processor to make the processor execute the following steps:

[0160] In response to the image defogging instruction, a to-be-processed image corresponding to the image defogging instruction is acquired;

[0161] The non-overlapping dark channels in the to-be-processed image are searched layer by layer to obtain an estimation value of the atmosphere light;

[0162] Based on a preset double-constraint strategy, a transmittance function in the to-be-processed image is acquired and energy minimization and guided filtering are performed to obtain an optimized transmittance;

[0163] The estimation value of the atmosphere light and the optimized transmittance are input into an image physical model corresponding to the to-be-processed image to obtain a defogging image corresponding to the to-be-processed image.

[0164] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer readable storage media that can store program codes.

[0165] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0166] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic. For example, the division of the units is merely a logical function division. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In this way, the inventive idea can be implemented.

[0167] The steps in the method embodiments of the present application can be adjusted, combined and deleted in sequence according to actual needs. The units in the apparatus embodiments of the present application can be combined, divided and deleted according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0168] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application, essentially or in the contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0169] The above describes only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image dehazing method, characterized in that, The image dehazing method includes: In response to an image dehazing command, obtain the image to be processed corresponding to the image dehazing command; A hierarchical search is performed on the non-overlapping dark channels in the image to be processed to obtain atmospheric light estimates. The transmittance function in the image to be processed is obtained based on a preset dual-constraint strategy, and energy minimization and guided filtering are performed to obtain the optimized transmittance; the dual-constraint strategy includes piecewise smoothing constraint conditions and scene radiation constraint conditions. The atmospheric light estimate and the optimized transmittance are input into the image physics model corresponding to the image to be processed to obtain the dehazed image corresponding to the image to be processed; the image physics model corresponding to the image to be processed is I(x) = J(x)t(x) - A(1-t(x)), where I(x) represents the image to be processed, x is the pixel coordinate, J(x) is the haze-free image to be restored, t(x) is the atmospheric transmittance, and A is the global atmospheric light; The step of performing a layered search on the non-overlapping dark channels in the image to be processed to obtain atmospheric light estimates includes: Obtain the non-overlapping dark channel based on the image to be processed; By performing a quadtree hierarchical search on the non-overlapping dark channels in the image to be processed, the final atmospheric light candidate region is obtained; The color vector with the shortest distance to the preset pixel value is obtained from the final atmospheric light candidate region as the atmospheric light estimate.

2. The method according to claim 1, characterized in that, The step of obtaining the non-overlapping dark channel based on the image to be processed includes: The image to be processed is divided into multiple non-overlapping image sub-blocks of the same size, and the minimum pixel value in each image sub-block is used as the pixel value of all pixels in the corresponding image sub-block to obtain the non-overlapping dark channel corresponding to each image sub-block.

3. The method according to claim 1, characterized in that, The method involves obtaining the transmittance function in the image to be processed based on a preset dual-constraint strategy, performing energy minimization and guided filtering to obtain optimized transmittance, including: Obtain the piecewise smoothing constraint and the scene radiation constraint in the dual constraint strategy, and construct an energy function corresponding to the image to be processed based on the piecewise smoothing constraint and the scene radiation constraint. The energy function corresponding to the image to be processed is used as the transmittance function to minimize the energy, and a rough estimate of the transmittance is obtained. The optimized transmittance is obtained by guiding filtering the coarsely estimated transmittance.

4. The method according to claim 3, characterized in that, The step of minimizing the energy function corresponding to the image to be processed as the transmittance function to obtain a rough estimate of transmittance includes: The energy function is minimized based on the α-expansion strategy to obtain the coarse estimate of transmittance.

5. The method according to claim 3, characterized in that, The step of obtaining the optimized transmittance by performing guided filtering on the coarsely estimated transmittance includes: The coarsely estimated transmittance is then subjected to edge-preserving smoothing and guided filtering to obtain the optimized transmittance.

6. The method according to any one of claims 1-5, characterized in that, The step of inputting the atmospheric light estimate and the optimized transmittance into the image physics model corresponding to the image to be processed to obtain the dehazed image corresponding to the image to be processed includes: Obtain the image physical model corresponding to the image to be processed, and adjust the image physical model based on the preset lower limit transmittance to obtain the adjusted image physical model. The atmospheric light estimate and the optimized transmittance are input into the adjusted image physical model to obtain a dehazed image corresponding to the image to be processed.

7. An image dehazing device, characterized in that, include: An initial image acquisition unit is used to acquire an image to be processed corresponding to an image dehazing instruction in response to the image dehazing instruction. Atmospheric light estimation unit is used to perform a layered search on the non-overlapping dark channels in the image to be processed to obtain atmospheric light estimation values. The transmittance optimization unit is used to obtain the transmittance function in the image to be processed based on a preset dual-constraint strategy and perform energy minimization and guided filtering to obtain the optimized transmittance; the dual-constraint strategy includes piecewise smoothing constraint conditions and scene radiation constraint conditions. The dehazing unit is used to input the atmospheric light estimate and the optimized transmittance into the image physical model corresponding to the image to be processed, so as to obtain a dehazed image corresponding to the image to be processed; the image physical model corresponding to the image to be processed is I(x) = J(x)t(x) - A(1-t(x)), where I(x) represents the image to be processed, x is the pixel coordinate, J(x) is the haze-free image to be restored, t(x) is the atmospheric transmittance, and A is the global atmospheric light; The atmospheric light estimation value acquisition unit is specifically used for: Obtain the non-overlapping dark channel based on the image to be processed; By performing a quadtree hierarchical search on the non-overlapping dark channels in the image to be processed, the final atmospheric light candidate region is obtained; The color vector with the shortest distance to the preset pixel value is obtained from the final atmospheric light candidate region as the atmospheric light estimate.

8. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, can implement the method as described in any one of claims 1-6.

Images