Image enhancement method, device and robot based on structure perception

Abstract

The present invention relates to the field of computer vision technology and provides a method, device, and robot for image enhancement based on structure perception. The method, based on the Retinex model, decomposes the original image into an illumination component and a reflection component, and constructs a variational model for image enhancement based on structure perception based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component. Through a semi-decoupled decomposition method, the illumination map and the reflection map are determined successively in each iteration by converting them into a least squares problem. This greatly reduces the difficulty of solving the final illumination map and the final reflection map, thereby improving the efficiency of image enhancement. In addition, the method, with the help of the illumination first-order gradient and the reflection first-order gradient, can respectively achieve structural perception of the illumination map and the reflection map, thereby improving the accuracy of the final illumination map and the final reflection map.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image enhancement method, device and robot based on structure perception. Background Art

[0002] With the continuous development of image processing in recent years, using images to obtain information has become an important means. For example, high-quality images are used for object detection. However, the image acquisition process is inevitably limited by the lighting environment and imaging technology, such as backlight, low light, uneven lighting, and insufficient exposure time. This can result in a low contribution rate of the resulting image.

[0003] Existing low-brightness image enhancement algorithms typically achieve this through simple constraints on image gradients, often resulting in artifacts and loss of detail. While this approach achieves image enhancement, it introduces other factors that affect image quality, resulting in a low contribution rate for the resulting image, making it unsuitable for subsequent applications such as object detection. Summary of the Invention

[0004] The present invention provides an image enhancement method, device and robot based on structure perception, which are used to solve the defects in the prior art.

[0005] The present invention provides an image enhancement method based on structure perception, comprising:

[0006] Based on the Retinex model, the original image is decomposed into an illumination component and a reflection component, and a structure-aware image enhancement variational model is constructed based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component;

[0007] Performing a semi-decoupled decomposition on the image enhancement variational model, constructing an illumination estimation model based on the illumination components, the original illumination map, and the first-order illumination gradient, converting the illumination estimation model into a first least squares problem, solving the first least squares problem, and obtaining an initial illumination map;

[0008] constructing a reflection estimation model based on the initial illumination map, the reflection component, the original image, and the first-order gradient of the reflection, converting the reflection estimation model into a second least squares problem, and solving the second least squares problem to obtain an initial reflection map;

[0009] Iteratively solving the first least squares problem and the second least squares problem simultaneously to obtain a final illumination map and a final reflection map, and determining an enhanced image corresponding to the original image based on the final illumination map and the final reflection map;

[0010] The original illumination is determined based on the maximum pixel value of the original image in all channels.

[0011] According to a structure-aware image enhancement method provided by the present invention, the image enhancement variational model includes a fidelity term, an illumination structure-aware regularization term, and a reflection structure-aware regularization term;

[0012] The fidelity item is determined based on the illumination component and the reflection component, and is used to maintain similarity between a product of the illumination component and the reflection component and the original image;

[0013] The illumination structure perception regularization term is determined based on the illumination first-order gradient, the L2 norm of the illumination first-order gradient, and the Gaussian filtering result of the L2 norm of the illumination first-order gradient, and is used to perform piecewise smoothing on the illumination component;

[0014] The reflection structure-aware regularization term is determined based on the reflection first-order gradient, the L2 norm of the reflection first-order gradient, and the Gaussian filtering result of the L2 norm of the reflection first-order gradient, and is used to maintain the structure of the reflection component.

[0015] According to the structure-aware image enhancement method provided by the present invention, the illumination structure-aware regularization term and the reflection structure-aware regularization term are both in the form of L1 norm;

[0016] Accordingly,

[0017] Converting the illumination estimation model into a first least squares problem includes:

[0018] Converting the illumination structure perception regularization term from an L1 norm form to an L2 norm form, and converting the illumination estimation model constructed based on the illumination structure perception regularization term in the L2 norm form into the first least squares problem;

[0019] Converting the reflection estimation model into a second least squares problem includes:

[0020] The reflection structure perception regularization term is converted from an L1 norm form to an L2 norm form, and the reflection estimation model constructed based on the reflection structure perception regularization term in the L2 norm form is converted into the second least squares problem.

[0021] According to a structure-aware image enhancement method provided by the present invention, the illumination structure-aware regularization term is expressed in the following form: ;

[0022] in, represents the first-order gradient of the illumination, represents the L2 norm of the first-order gradient of the illumination, The standard deviation is Gaussian filter, Represents the texture coefficient, which is a constant. represents the L1 norm, It is an exponential operation with the natural constant e as the base;

[0023] The reflective structure-aware regularization term is expressed in the following form: ;

[0024] in, represents the first-order gradient of the reflection, represents the L2 norm of the reflected first-order gradient.

[0025] According to a structure-aware image enhancement method provided by the present invention, the first least squares problem and the second least squares problem are simultaneously iteratively solved to obtain a final illumination map and a final reflection map, including:

[0026] For a current iteration round, deriving the first least squares problem with respect to the illumination component, and taking the illumination component value when the derivative of the first least squares problem is 0 as the current illumination map of the current iteration round;

[0027] Derivative the reflection component of the second least squares problem, and take the reflection component value when the derivative of the second least squares problem is 0 as the current reflection map of the current iteration round;

[0028] Determining whether to stop iteration based on an F-norm of a difference map between the current illumination map and a previous illumination map of the current illumination map, an F-norm of the previous illumination map, an F-norm of a difference map between the current reflection map and a previous reflection map of the current reflection map, and an F-norm of the previous reflection map;

[0029] Alternatively, whether to stop the iteration is determined based on a predetermined maximum number of iterations.

[0030] According to a structure-aware image enhancement method provided by the present invention, determining an enhanced image corresponding to the original image based on the final illumination map and the final reflection map includes:

[0031] Performing gamma correction on the final illumination map to obtain a correction result;

[0032] The correction result is element-wise multiplied by the final reflection map to obtain the enhanced image.

[0033] The present invention also provides an image enhancement device based on structure perception, comprising:

[0034] a model building module for decomposing an original image into an illumination component and a reflection component based on a Retinex model, and building a structure-aware image enhancement variational model based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component;

[0035] an illumination solution module, configured to perform a semi-decoupled decomposition on the image enhancement variational model, construct an illumination estimation model based on the illumination components, the original illumination map, and the first-order illumination gradient, convert the illumination estimation model into a first least squares problem, solve the first least squares problem, and obtain an initial illumination map;

[0036] a reflection solving module, configured to construct a reflection estimation model based on the initial illumination map, the reflection component, the original image, and the first-order gradient of the reflection, convert the reflection estimation model into a second least squares problem, and solve the second least squares problem to obtain an initial reflection map;

[0037] an image enhancement module, configured to iteratively solve the first least squares problem and the second least squares problem simultaneously to obtain a final illumination map and a final reflection map, and determine an enhanced image corresponding to the original image based on the final illumination map and the final reflection map;

[0038] The original illumination is determined based on the maximum pixel value of the original image in all channels.

[0039] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image enhancement method based on structure perception as described above is implemented.

[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described structure-aware image enhancement methods.

[0041] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-described structure-aware image enhancement methods.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention provides an image enhancement method, device and robot based on structure perception. The method first decomposes the original image into an illumination component and a reflection component based on a Retinex model, and constructs an image enhancement variational model based on structure perception based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component and the reflection first-order gradient corresponding to the reflection component; thereafter, the image enhancement variational model is semi-decoupled and decomposed, and an illumination estimation model is constructed based on the illumination component, the original illumination map and the illumination first-order gradient, and the illumination estimation model is converted into a first least squares problem, and the first least squares problem is solved to obtain an initial illumination map; based on the initial illumination map, the reflection component, the original image and the reflection first-order gradient, a reflection estimation model is constructed, and the reflection estimation model is converted into a second least squares problem, and the second least squares problem is solved to obtain an initial reflection map; finally, the first least squares problem and the second least squares problem are simultaneously iteratively solved to obtain a final illumination map and a final reflection map, and based on the final illumination map and the final reflection map, an enhanced image corresponding to the original image is determined. This method uses a semi-decoupled decomposition approach, transforming each iteration into a least-squares problem to determine the illumination map and reflectance map. This significantly reduces the difficulty of solving the final illumination and reflectance maps, improving the efficiency of image enhancement. Furthermore, by leveraging the first-order gradients of illumination and reflectance, this method achieves structural perception of the illumination and reflectance maps, respectively, thereby improving the accuracy of the final illumination and reflectance maps. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on the drawings in the following description without any creative work.

[0045] Figure 1 1 is a flow chart of the structure-aware image enhancement method provided by the present invention;

[0046] Figure 2 is one of the original images in the structure-aware image enhancement method provided by the present invention;

[0047] Figure 3 yes Figure 2 The corresponding enhanced image;

[0048] Figure 4 It is the second original image in the structure-aware image enhancement method provided by the present invention;

[0049] Figure 5 yes Figure 4 The corresponding enhanced image;

[0050] Figure 6 This is the third original image in the structure-aware image enhancement method provided by the present invention;

[0051] Figure 7 yes Figure 6 The corresponding enhanced image;

[0052] Figure 8 This is the fourth original image in the structure-aware image enhancement method provided by the present invention;

[0053] Figure 9 yes Figure 8 The corresponding enhanced image;

[0054] Figure 10 Schematic diagram of the structure of the image enhancement device based on structure perception provided by the present invention;

[0055] Figure 11 It is a structural schematic diagram of the robot provided by the present invention;

[0056] Figure 12 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0057] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0058] Although existing low-brightness image enhancement algorithms achieve image enhancement, they introduce other factors that affect image quality, resulting in a low contribution rate for the resulting image, making it unsuitable for subsequent applications such as target detection. Based on this, an embodiment of the present invention provides an image enhancement method based on structure perception.

[0059] Figure 1 FIG. 1 is a flow chart of an image enhancement method based on structure perception provided in an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0060] S1, based on the Retinex model, decomposes the original image into an illumination component and a reflection component, and constructs a structure-aware image enhancement variational model based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component;

[0061] S2, performing semi-decoupled decomposition on the image enhancement variational model, constructing an illumination estimation model based on the illumination components, the original illumination map, and the first-order illumination gradient, converting the illumination estimation model into a first least squares problem, solving the first least squares problem, and obtaining an initial illumination map;

[0062] S3: constructing a reflection estimation model based on the initial illumination map, the reflection component, the original image, and the first-order reflection gradient, converting the reflection estimation model into a second least squares problem, and solving the second least squares problem to obtain an initial reflection map;

[0063] S4, iteratively solving the first least squares problem and the second least squares problem simultaneously to obtain a final illumination map and a final reflection map, and determining an enhanced image corresponding to the original image based on the final illumination map and the final reflection map;

[0064] The original illumination is determined based on the maximum pixel value of the original image in all channels.

[0065] Specifically, the image enhancement method based on structure perception provided in the embodiment of the present invention is executed by an image enhancement device based on structure perception, which can be configured in a computer. The computer can be a local computer, an offline computer or a cloud computer. The local computer can be a computer, a tablet, etc., and the offline computer can be an industrial control board, etc., which is not specifically limited here.

[0066] First, execute step S1 to decompose the original image into illumination component and reflection component based on the Retinex model. The Retinex model is: (1)

[0067] Among them, I is the original image, L is the illumination component, and R is the reflection component.

[0068] It is understood that the original image is a color image captured by an image acquisition device and is a low-brightness image that needs to be enhanced. The image acquisition device can be a camera that can be mounted on a robot. The illumination component represents the illumination changes of the original image. Here, it is assumed that the illumination component is piecewise smooth and only contains the main structures in the original image. The reflection component describes the reflection characteristics of the objects contained in the original image to the illumination. The reflection component contains the structure and texture in the original image. Operator It stands for element-wise multiplication, that is, the illumination component and the reflection component are multiplied element-wise to get the original image.

[0069] In order to estimate accurate illumination maps and reflectance maps, an image enhancement variational model based on structure perception is constructed in an embodiment of the present invention, which can achieve piecewise smoothing of the illumination component and structural preservation of the reflectance component.

[0070] The image enhancement variational model may include a fidelity term, an illumination structure-aware regularization term, and a reflection structure-aware regularization term. The fidelity term may be determined by the illumination component and the reflection component, and is used to maintain the similarity between the product of the illumination component and the reflection component and the original image. The illumination structure-aware regularization term may be determined based on the Gaussian filtering results of the illumination first-order gradient, the L2 norm of the illumination first-order gradient, and the L2 norm of the illumination first-order gradient, and is used to perform piecewise smoothing of the illumination component. The reflection structure-aware regularization term may be determined based on the Gaussian filtering results of the reflection first-order gradient, the L2 norm of the reflection first-order gradient, and the L2 norm of the reflection first-order gradient, and is used to maintain the structure of the reflection component, and a reflection map with rich structure may be solved. Both the illumination structure-aware regularization term and the reflection structure-aware regularization term may be in the form of the L1 norm.

[0071] The structure-aware image enhancement variational model can be characterized by the following minimization problem: (2)

[0072] in, For authenticity, is the illumination structure perception regularization term, is the reflective structure-aware regularization term. are the balance coefficients that control the importance of the illumination structure perception regularization term and the reflection structure perception regularization term, which can be constants. For example, they can be . and are respectively Norm and norm. Is a first-order gradient operator, including horizontal gradient and vertical gradient . To represent the first-order gradient of illumination, represents the reflected first-order gradient, represents the L2 norm of the first-order gradient of illumination, represents the L2 norm of the reflected first-order gradient, and . Represents the texture coefficient, which is used to perceive texture and can be , represents the exponential operation with the natural constant e as the base, The standard deviation is Gaussian filter, operator Represents a convolution operation. The Gaussian filtering result of the L2 norm of the first-order gradient of illumination, Represents the Gaussian filtering result of the L2 norm of the reflected first-order gradient.

[0073] Then, step S2 is executed to perform semi-decoupling decomposition on the image enhancement variational model, and to construct an illumination estimation model based on the illumination component, the original illumination map, and the first-order illumination gradient.

[0074] For solving the initial illumination map, we can extract the terms related to the illumination components in formula (2). In order to avoid involving unknown reflection components in the illumination map solution process, we use semi-decoupled decomposition to estimate the illumination map using the illumination components and the original illumination map. The constructed illumination estimation model can be expressed as: (3)

[0075] in, is the original light map.

[0076] The original light map can be determined by the maximum pixel value of the original image in all channels. If the original image is a color image, there are three RGB channels, then the original light map can be expressed as: (4)

[0077] Where x is each pixel in the original image, Represents the pixel value of pixel x of channel c of the original image, is the pixel value of pixel x in the original light map.

[0078] Afterwards, for the convenience of calculation, the illumination estimation model can be converted into a first least squares problem, and the first least squares problem is solved to obtain an initial illumination map.

[0079] Similarly, for solving the initial reflection map, we can extract the terms related to the illumination component in formula (2) and construct a reflection estimation model, as shown in formula (5): (5)

[0080] To facilitate calculation, the reflection estimation model can be converted into a second least squares problem. The second least squares problem is solved to obtain the initial reflection map. This process is consistent with the process of solving the initial illumination map and will not be repeated here.

[0081] Finally, step S3 is executed to iteratively solve the first and second least squares problems simultaneously. That is, in each iteration, the first least squares problem is solved first, and then the second least squares problem is solved using the resulting illumination map until the iteration stop condition is met, resulting in the final illumination map and final reflection map. The final illumination map accurately represents the light and dark variations in the original image, and the final reflection map accurately reflects the rich structural textures in the original image.

[0082] Afterwards, the final illumination map and the final reflection map are used to determine the enhanced image corresponding to the original image. For example, the enhanced image can be obtained by directly multiplying the final illumination map and the final reflection map element-wise.

[0083] An image enhancement method based on structure perception is provided in an embodiment of the present invention. First, based on the Retinex model, the original image is decomposed into an illumination component and a reflection component, and a structure-aware image enhancement variational model is constructed based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component. Thereafter, the image enhancement variational model is semi-decoupled and decomposed. An illumination estimation model is constructed based on the illumination component, the original illumination map, and the illumination first-order gradient. The illumination estimation model is converted into a first least squares problem, and the first least squares problem is solved to obtain an initial illumination map. Finally, based on the initial illumination map of the illumination component, the reflection component, the original image, and the reflection first-order gradient, a reflection estimation model is constructed. The reflection estimation model is converted into a second least squares problem, and the second least squares problem is solved to obtain an initial reflection map. Finally, the first least squares problem and the second least squares problem are simultaneously iteratively solved to obtain a final illumination map and a final reflection map, and based on the final illumination map and the final reflection map, an enhanced image corresponding to the original image is determined. This method uses a semi-decoupled decomposition approach, transforming each iteration into a least-squares problem to determine the illumination map and reflectance map. This significantly reduces the difficulty of solving the final illumination and reflectance maps, improving the efficiency of image enhancement. Furthermore, by leveraging the first-order gradients of illumination and reflectance, this method achieves structural perception of the illumination and reflectance maps, respectively, thereby improving the accuracy of the final illumination and reflectance maps.

[0084] This method enhances the original images captured by image acquisition devices, improving the visual environment in which robots operate. This allows robots equipped with image acquisition devices to operate efficiently in low-light environments, providing strong technical support for intelligent manufacturing and industrial production. Furthermore, this method can be applied to various types of robots, improving their robustness to the environment.

[0085] On the basis of the above embodiment, in the structure-aware image enhancement method provided in the embodiment of the present invention, the illumination structure-aware regularization term and the reflection structure-aware regularization term are both in the form of L1 norm; accordingly,

[0086] Converting the illumination estimation model into a first least squares problem includes:

[0087] Converting the illumination structure perception regularization term from an L1 norm form to an L2 norm form, and converting the illumination estimation model constructed based on the illumination structure perception regularization term in the L2 norm form into the first least squares problem;

[0088] Converting the reflection estimation model into a second least squares problem includes:

[0089] The reflection structure perception regularization term is converted from an L1 norm form to an L2 norm form, and the reflection estimation model constructed based on the reflection structure perception regularization term in the L2 norm form is converted into the second least squares problem.

[0090] Specifically, as shown in formula (2), the illumination structure perception regularization term and the reflection structure perception regularization term are both in the form of L1 norm. However, the L1 norm is difficult to solve. In order to efficiently solve the initial illumination map and the initial reflection map, the limit idea is adopted here to solve the L1 norm equivalent to the L2 norm.

[0091] That is, the illumination structure perception regularization term is converted from the L1 norm form to the L2 norm form, and the following formula (6) is obtained: (6)

[0092] in, is a small constant to avoid the denominator being zero, usually set to 0.001.

[0093] Since formula (6) decomposes the illumination structure perception regularization term into a quadratic term and weights , so formula (3) can be rewritten to obtain the first least squares problem, as shown in formula (7): (7)

[0094] Furthermore, during the simultaneous iterative solution of the first and second least squares problems to obtain the final illumination map and final reflectance map, for the current iteration, the first least squares problem is differentiated with respect to the illumination component, and the illumination component value when the derivative of the first least squares problem is 0 is taken as the current illumination map for the current iteration. This process also applies to solving the initial illumination map.

[0095] It is worth noting that in each iteration are all known variables and are calculated from the illumination components of the previous round. For the first round of iteration, can be calculated directly.

[0096] Current Light Map It can be expressed as: (8)

[0097] Where d represents the dimension, including the horizontal dimension h and the vertical dimension v. 1 is the identity matrix of the appropriate size, are the diagonal matrices of the corresponding variables. is the Toeplitz matrix with the forward differencing discrete gradient operator, is the d-dimensional representation of the Toeplitz matrix, is the horizontal dimension representation of the Toeplitz matrix, is the vertical dimension of the Toeplitz matrix.

[0098] Similarly, converting the reflective structure perception regularization term from the L1 norm form to the L2 norm form, we have the following formula (9): (9)

[0099] Since formula (9) decomposes the reflection structure-aware regularization term into a quadratic term and weights , so formula (5) can be rewritten to obtain the second least squares problem, as shown in formula (10): (10)

[0100] Furthermore, while simultaneously iteratively solving the first and second least squares problems to obtain the final illumination map and final reflectance map, for the current iteration, the second least squares problem is differentiated with respect to the reflectance component, and the reflectance component value when the derivative of the second least squares problem is 0 is taken as the current reflectance map for the current iteration. This process also applies to solving the initial reflectance map.

[0101] It is worth noting that in each iteration are all known variables and are calculated from the reflection components of the previous round. For the first round of iteration, It can be calculated directly.

[0102] Current reflection map It can be expressed as: (11)

[0103] in, yes The diagonal matrix of .

[0104] Based on the above embodiments, the structure-aware image enhancement method provided in the embodiments of the present invention simultaneously iteratively solves the first least squares problem and the second least squares problem to obtain the final illumination map and the final reflection map. After obtaining the current illumination map and the current reflection map of the current iteration round, there are two conditions for determining whether to stop the iteration. The iteration can be stopped if one of the conditions is met.

[0105] Condition 1: The F-norm of the difference map between the current illumination map and the illumination map before the current illumination map, the F-norm of the previous illumination map, the F-norm of the difference map between the current reflection map and the reflection map before the current reflection map, and the F-norm of the previous reflection map can be used to determine whether the iteration should stop. In other words, determine whether Formula (12) and Formula (13) hold. If they do, the iteration stops.

[0106] (12) (13)

[0107] in, represents the F norm, represents the current number of iterations, k-1 represents the previous number of iterations, and K represents the maximum number of iterations. Represents the current reflection map, represents the previous reflection image, Represents the current light map, Represents the previous light map.

[0108] Among them, the parameters can be set to .

[0109] Condition 2: Use the predetermined maximum number of iterations K to determine whether the iteration should stop, that is, determine whether the current number of iterations reaches the maximum number of iterations K. If it reaches the maximum number of iterations K, the iteration stops.

[0110] In the embodiment of the present invention, a condition for stopping iteration is given, which reduces the amount of calculation and prevents redundant iterations while ensuring the quality of the final illumination map and the final reflection map.

[0111] Based on the above embodiment, the structure-aware image enhancement method provided in the embodiment of the present invention, wherein determining the enhanced image corresponding to the original image based on the solution result of the illumination component and the solution result of the reflection component includes:

[0112] Performing gamma correction on the solution result of the illumination component to obtain a correction result;

[0113] The rectification result and the solution result of the reflection component are multiplied element by element to obtain the enhanced image.

[0114] Specifically, in embodiments of the present invention, when determining an enhanced image, gamma correction can be performed on the final illumination map to obtain a correction result. The gamma exponent used in the gamma correction can be set as needed, for example, 1 / 2.2, or other values, which are not specifically limited here.

[0115] Afterwards, the rectification result can be element-wise multiplied with the final reflectance map to obtain the enhanced image.

[0116] The enhanced image can be expressed as: (14)

[0117] in, To enhance the image, is the Gamma index.

[0118] In the embodiment of the present invention, gamma correction is used to adjust the final illumination map to obtain an enhanced image. This can make the enhanced image colorful and object features free of artifacts and halos.

[0119] Figure 2 、 Figure 4 、 Figure 6 and Figure 8 All are original images. Figure 3 、 Figure 5 、 Figure 7 and Figure 9 are the corresponding enhanced images respectively. Figure 2 and Figure 3 Make comparisons, Figure 4 and Figure 5 Make comparisons, Figure 6 and Figure 7 Make comparisons, Figure 8 and Figure 9 By comparison, it can be seen that the enhanced image has higher brightness than the original image, and the brightness of the entire image is uniform.

[0120] like Figure 10 As shown, based on the above embodiment, an embodiment of the present invention provides an image enhancement device based on structure perception, including:

[0121] The model construction module 101 is used to decompose the original image into an illumination component and a reflection component based on the Retinex model, and to construct an image enhancement variational model based on structure perception based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component;

[0122] an illumination solution module 102 for performing a semi-decoupled decomposition on the image enhancement variational model, constructing an illumination estimation model based on the illumination components, the original illumination map, and the first-order illumination gradient, converting the illumination estimation model into a first least squares problem, and solving the first least squares problem to obtain an initial illumination map;

[0123] a reflection solving module 103 for constructing a reflection estimation model based on the initial illumination map, the reflection component, the original image, and the first-order gradient of the reflection, converting the reflection estimation model into a second least squares problem, and solving the second least squares problem to obtain an initial reflection map;

[0124] An image enhancement module 104 is configured to iteratively solve the first least squares problem and the second least squares problem simultaneously to obtain a final illumination map and a final reflection map, and determine an enhanced image corresponding to the original image based on the final illumination map and the final reflection map;

[0125] The original illumination is determined based on the maximum pixel value of the original image in all channels.

[0126] On the basis of the above embodiment, the image enhancement device based on structure perception provided in the embodiment of the present invention comprises the image enhancement variational model including a fidelity term, an illumination structure perception regularization term and a reflection structure perception regularization term;

[0127] The fidelity item is determined based on the illumination component and the reflection component, and is used to maintain similarity between a product of the illumination component and the reflection component and the original image;

[0128] The illumination structure perception regularization term is determined based on the illumination first-order gradient, the L2 norm of the illumination first-order gradient, and the Gaussian filtering result of the L2 norm of the illumination first-order gradient, and is used to perform piecewise smoothing on the illumination component;

[0129] The reflection structure-aware regularization term is determined based on the reflection first-order gradient, the L2 norm of the reflection first-order gradient, and the Gaussian filtering result of the L2 norm of the reflection first-order gradient, and is used to maintain the structure of the reflection component.

[0130] On the basis of the above embodiment, in the image enhancement device based on structure perception provided in the embodiment of the present invention, the illumination structure perception regularization term and the reflection structure perception regularization term are both in the form of L1 norm; accordingly,

[0131] The illumination solution module is specifically used to:

[0132] Converting the illumination structure perception regularization term from an L1 norm form to an L2 norm form, and converting the illumination estimation model constructed based on the illumination structure perception regularization term in the L2 norm form into the first least squares problem;

[0133] The reflection solution module is specifically used to:

[0134] The reflection structure perception regularization term is converted from an L1 norm form to an L2 norm form, and the reflection estimation model constructed based on the reflection structure perception regularization term in the L2 norm form is converted into the second least squares problem.

[0135] On the basis of the above embodiments, in the structure-aware image enhancement device provided in the embodiments of the present invention, the illumination structure-aware regularization term is expressed in the following form: ;

[0136] in, represents the first-order gradient of the illumination, represents the L2 norm of the first-order gradient of the illumination, The standard deviation is Gaussian filter, Represents the texture coefficient, which is a constant. represents the L1 norm;

[0137] The reflective structure-aware regularization term is expressed in the following form: ;

[0138] in, represents the first-order gradient of the reflection, represents the L2 norm of the reflected first-order gradient.

[0139] On the basis of the above embodiments, in the structure-aware image enhancement device provided in the embodiments of the present invention, the image enhancement module is specifically configured to:

[0140] For a current iteration round, deriving the first least squares problem with respect to the illumination component, and taking the illumination component value when the derivative of the first least squares problem is 0 as the current illumination map of the current iteration round;

[0141] Derivative the reflection component of the second least squares problem, and take the reflection component value when the derivative of the second least squares problem is 0 as the current reflection map of the current iteration round;

[0142] Determining whether to stop iteration based on an F-norm of a difference map between the current illumination map and a previous illumination map of the current illumination map, an F-norm of the previous illumination map, an F-norm of a difference map between the current reflection map and a previous reflection map of the current reflection map, and an F-norm of the previous reflection map;

[0143] Alternatively, whether to stop the iteration is determined based on a predetermined maximum number of iterations.

[0144] On the basis of the above embodiments, in the structure-aware image enhancement device provided in the embodiments of the present invention, the image enhancement module is further specifically configured to:

[0145] Performing gamma correction on the final illumination map to obtain a correction result;

[0146] The correction result is element-wise multiplied by the final reflection map to obtain the enhanced image.

[0147] Specifically, the functions of each module in the structure-aware image enhancement device provided in the embodiment of the present invention correspond one-to-one to the operation process of each step in the above-mentioned method embodiment, and the effects achieved are also consistent. Please refer to the above-mentioned embodiment for details, and no further details will be given in the embodiment of the present invention.

[0148] like Figure 11 As shown, based on the above embodiment, an embodiment of the present invention further provides a robot, including a controller 111, a robot body 112 and an image acquisition device 113 mounted on the robot body 112, wherein the controller 111 is connected to the robot body 112 and the image acquisition device 113;

[0149] The image acquisition device 113 is used to collect original images of the robot body 112 during the working process and send the original images to the controller 111;

[0150] The controller 111 is used to determine an enhanced image of the original image based on the structure perception-based image enhancement method provided in the above embodiments, and control the robot body 112 based on the enhanced image so that the robot body 112 performs corresponding operations.

[0151] Specifically, the robot controller can enhance the original images captured by the image acquisition device, improving the robot's visual environment. This allows the robot equipped with the image acquisition device to operate efficiently in low-light environments, providing strong technical support for intelligent manufacturing and industrial production. Furthermore, this method can be applied to various types of robots, improving their robustness to the environment.

[0152] Figure 12 An example of a physical structure diagram of an electronic device is shown below. Figure 12As shown, the electronic device may include: a processor (Processor) 121, a communication interface (Communications Interface) 122, a memory (Memory) 123 and a communication bus 124, wherein the processor 121, the communication interface 122 and the memory 123 communicate with each other through the communication bus 124. The processor 121 can call the logic instructions in the memory 123 to execute the image enhancement method based on structure perception provided in the above embodiments, the method comprising: decomposing the original image into an illumination component and a reflection component based on the Retinex model, and constructing an image enhancement variational model based on structure perception based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component and the reflection first-order gradient corresponding to the reflection component; performing semi-decoupling decomposition on the image enhancement variational model, constructing an illumination estimation model based on the illumination component, the original illumination map and the illumination first-order gradient, and converting the illumination estimation model into a first least squares problem. problem, solve the first least squares problem to obtain an initial illumination map; construct a reflection estimation model based on the initial illumination map, the reflection component, the original image and the first-order gradient of the reflection, and convert the reflection estimation model into a second least squares problem, solve the second least squares problem to obtain an initial reflection map; iteratively solve the first least squares problem and the second least squares problem simultaneously to obtain a final illumination map and a final reflection map, and determine an enhanced image corresponding to the original image based on the final illumination map and the final reflection map; wherein the original illumination is determined based on the maximum pixel value of the original image in all channels.

[0153] Furthermore, the logic instructions in the aforementioned memory 123 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, an offline device, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0154] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image enhancement method based on structure perception provided in the above embodiments, the method including: decomposing the original image into an illumination component and a reflection component based on a Retinex model, and constructing an image enhancement variational model based on structure perception based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component; semi-decoupling decomposition of the image enhancement variational model, and constructing a structure-perception-based image enhancement variational model based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the reflection component, and the reflection first-order gradient corresponding to the reflection component. An illumination estimation model is constructed, and the illumination estimation model is converted into a first least squares problem, and the first least squares problem is solved to obtain an initial illumination map; based on the initial illumination map, the reflection component, the original image, and the first-order gradient of the reflection, a reflection estimation model is constructed, and the reflection estimation model is converted into a second least squares problem, and the second least squares problem is solved to obtain an initial reflection map; the first least squares problem and the second least squares problem are simultaneously iteratively solved to obtain a final illumination map and a final reflection map, and based on the final illumination map and the final reflection map, an enhanced image corresponding to the original image is determined; wherein, the original illumination is determined based on the maximum pixel value of the original image in all channels.

[0155] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the image enhancement method based on structure perception provided in the above embodiments, the method comprising: decomposing the original image into an illumination component and a reflection component based on a Retinex model, and constructing an image enhancement variational model based on structure perception based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component; performing semi-decoupling decomposition on the image enhancement variational model, constructing an illumination estimation model based on the illumination component, the original illumination map, and the illumination first-order gradient, and converting the illumination component into an illumination estimation model; The estimation model is converted into a first least squares problem, and the first least squares problem is solved to obtain an initial illumination map; based on the initial illumination map, the reflection component, the original image and the first-order gradient of the reflection, a reflection estimation model is constructed, and the reflection estimation model is converted into a second least squares problem, and the second least squares problem is solved to obtain an initial reflection map; the first least squares problem and the second least squares problem are solved iteratively simultaneously to obtain a final illumination map and a final reflection map, and based on the final illumination map and the final reflection map, an enhanced image corresponding to the original image is determined; wherein, the original illumination is determined based on the maximum pixel value of the original image in all channels.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0157] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A structure-aware image enhancement method, characterized in that: include: Based on the Retinex model, the original image is decomposed into an illumination component and a reflection component, and a structure-aware image enhancement variational model is constructed based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component; Performing a semi-decoupled decomposition on the image enhancement variational model, constructing an illumination estimation model based on the illumination components, the original illumination map, and the first-order illumination gradient, converting the illumination estimation model into a first least squares problem, solving the first least squares problem, and obtaining an initial illumination map; constructing a reflection estimation model based on the initial illumination map, the reflection component, the original image, and the first-order gradient of the reflection, converting the reflection estimation model into a second least squares problem, and solving the second least squares problem to obtain an initial reflection map; Iteratively solving the first least squares problem and the second least squares problem simultaneously to obtain a final illumination map and a final reflection map, and determining an enhanced image corresponding to the original image based on the final illumination map and the final reflection map; The original illumination is determined based on the maximum pixel value of the original image in all channels; The image enhancement variational model includes a fidelity term, an illumination structure perception regularization term, and a reflection structure perception regularization term; The fidelity item is determined based on the illumination component and the reflection component, and is used to maintain similarity between a product of the illumination component and the reflection component and the original image; The illumination structure perception regularization term is determined based on the illumination first-order gradient, the L2 norm of the illumination first-order gradient, and the Gaussian filtering result of the L2 norm of the illumination first-order gradient, and is used to perform piecewise smoothing on the illumination component; The reflection structure perception regularization term is determined based on the reflection first-order gradient, the L2 norm of the reflection first-order gradient, and the Gaussian filtering result of the L2 norm of the reflection first-order gradient, and is used to maintain the structure of the reflection component; The illumination structure perception regularization term is expressed in the following form: ； in, represents the first-order gradient of the illumination, represents the L2 norm of the first-order gradient of the illumination, The standard deviation is Gaussian filter, Represents the texture coefficient, which is a constant. represents the L1 norm, It is an exponential operation with the natural constant e as the base; The reflective structure-aware regularization term is expressed in the following form: ； in, represents the first-order gradient of the reflection, represents the L2 norm of the reflected first-order gradient.

2. The structure-aware image enhancement method according to claim 1, wherein: The illumination structure perception regularization term and the reflection structure perception regularization term are both in the form of L1 norm; Accordingly, Converting the illumination estimation model into a first least squares problem includes: Converting the illumination structure perception regularization term from an L1 norm form to an L2 norm form, and converting the illumination estimation model constructed based on the illumination structure perception regularization term in the L2 norm form into the first least squares problem; Converting the reflection estimation model into a second least squares problem includes: The reflection structure perception regularization term is converted from an L1 norm form to an L2 norm form, and the reflection estimation model constructed based on the reflection structure perception regularization term in the L2 norm form is converted into the second least squares problem.

3. The structure-aware image enhancement method according to any one of claims 1 to 2, characterized in that: Iteratively solving the first least squares problem and the second least squares problem simultaneously to obtain a final illumination map and a final reflection map, including: For a current iteration round, deriving the first least squares problem with respect to the illumination component, and taking the illumination component value when the derivative of the first least squares problem is 0 as the current illumination map of the current iteration round; Derivative the reflection component of the second least squares problem, and take the reflection component value when the derivative of the second least squares problem is 0 as the current reflection map of the current iteration round; Determining whether to stop iteration based on an F-norm of a difference map between the current illumination map and a previous illumination map of the current illumination map, an F-norm of the previous illumination map, an F-norm of a difference map between the current reflection map and a previous reflection map of the current reflection map, and an F-norm of the previous reflection map; Alternatively, whether to stop the iteration is determined based on a predetermined maximum number of iterations.

4. The structure-aware image enhancement method according to any one of claims 1 to 2, characterized in that: The determining, based on the final illumination map and the final reflection map, an enhanced image corresponding to the original image includes: Performing gamma correction on the final illumination map to obtain a correction result; The correction result is element-wise multiplied by the final reflection map to obtain the enhanced image.

5. An image enhancement device based on structure perception, characterized in that: include: a model building module for decomposing an original image into an illumination component and a reflection component based on a Retinex model, and building a structure-aware image enhancement variational model based on the illumination component, the reflection component, the illumination first-order gradient corresponding to the illumination component, and the reflection first-order gradient corresponding to the reflection component; an illumination solution module, configured to perform a semi-decoupled decomposition on the image enhancement variational model, construct an illumination estimation model based on the illumination components, the original illumination map, and the first-order illumination gradient, convert the illumination estimation model into a first least squares problem, solve the first least squares problem, and obtain an initial illumination map; a reflection solving module, configured to construct a reflection estimation model based on the initial illumination map, the reflection component, the original image, and the first-order gradient of the reflection, convert the reflection estimation model into a second least squares problem, and solve the second least squares problem to obtain an initial reflection map; an image enhancement module, configured to iteratively solve the first least squares problem and the second least squares problem simultaneously to obtain a final illumination map and a final reflection map, and determine an enhanced image corresponding to the original image based on the final illumination map and the final reflection map; The original illumination is determined based on the maximum pixel value of the original image in all channels; The image enhancement variational model includes a fidelity term, an illumination structure perception regularization term, and a reflection structure perception regularization term; The fidelity item is determined based on the illumination component and the reflection component, and is used to maintain similarity between a product of the illumination component and the reflection component and the original image; The illumination structure perception regularization term is determined based on the illumination first-order gradient, the L2 norm of the illumination first-order gradient, and the Gaussian filtering result of the L2 norm of the illumination first-order gradient, and is used to perform piecewise smoothing on the illumination component; The reflection structure perception regularization term is determined based on the reflection first-order gradient, the L2 norm of the reflection first-order gradient, and the Gaussian filtering result of the L2 norm of the reflection first-order gradient, and is used to maintain the structure of the reflection component; The illumination structure perception regularization term is expressed in the following form: ； in, represents the first-order gradient of the illumination, represents the L2 norm of the first-order gradient of the illumination, The standard deviation is Gaussian filter, Represents the texture coefficient, which is a constant. represents the L1 norm, It is an exponential operation with the natural constant e as the base; The reflective structure-aware regularization term is expressed in the following form: ； in, represents the first-order gradient of the reflection, represents the L2 norm of the reflected first-order gradient.

6. A robot, characterized in that: It includes a controller, a robot body, and an image acquisition device mounted on the robot body, wherein the controller is connected to the robot body and the image acquisition device; The image acquisition device is used to acquire original images of the robot body during operation and send the original images to the controller; The controller is configured to determine an enhanced image of the original image based on the structure-aware image enhancement method according to any one of claims 1 to 4, and control the robot body based on the enhanced image.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the structure-aware image enhancement method according to any one of claims 1 to 4 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the structure-aware image enhancement method according to any one of claims 1 to 4 is implemented.

Images