Hybrid exposure processing method and device based on aesthetic evaluation, medium and product

Abstract

The invention provides a mixed exposure processing method and device based on aesthetic evaluation, a medium and a product. Comprising the following steps: performing feature mapping processing on to-be-corrected feature information according to mask information to obtain first overexposure feature information and first underexposure feature information; performing pooling convolution processing on the first overexposure feature information and the first underexposure feature information to obtain second overexposure feature information and second underexposure feature information; obtaining target overexposure feature information and target underexposure feature information according to the first overexposure feature information, the second overexposure feature information, the first underexposure feature information and the second underexposure feature information; obtaining fusion feature information according to the to-be-corrected feature information, the target overexposure feature information and the target underexposure feature information; and obtaining a target image according to the fused feature information and the to-be-corrected feature image. According to the method, the underexposure part and the overexposure part in the exposure image can be recognized and processed at the same time, the exposure correction efficiency is improved, and the impression experience of a user on the image is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a hybrid exposure processing method, device, medium and product based on aesthetic evaluation. Background Art

[0002] With the rapid development of photographic equipment in recent years, photography has become common. However, images taken under unbalanced ambient lighting conditions often suffer from overexposure and underexposure problems, resulting in severe structural distortion and quality degradation.

[0003] Existing exposure processing technologies usually use image processing algorithms to identify exposure problems in images, and use adjustment equipment to deal with exposure problems, or increase or decrease the exposure value of the image through exposure compensation or image fusion to obtain better exposure effects.

[0004] However, the existing exposure processing technology has a slow speed in recognizing image information and a large error in extracting exposure information, and is unable to accurately complete the recognition and correction of uneven exposure areas in the image. Summary of the invention

[0005] The present application provides a hybrid exposure processing method, device, medium and product based on aesthetic evaluation, which are used to solve technical problems such as low efficiency in identifying uneven exposure areas and poor correction processing technology in existing exposure correction technology.

[0006] In a first aspect, the present application provides a hybrid exposure processing method based on aesthetic evaluation, comprising: According to the overexposure mask information and the underexposure mask information, feature mapping processing is performed on the feature information to be corrected respectively to obtain first overexposure feature information and first underexposure feature information in the feature information to be corrected, the overexposure mask information is obtained according to the overexposed region image in the feature image to be corrected, the underexposure mask information is obtained according to the underexposed region image in the feature image to be corrected, and the feature information to be corrected is feature information of the feature image to be corrected; Performing pooling convolution processing on the first overexposure feature information and the first underexposure feature information respectively to obtain second overexposure feature information and second underexposure feature information; Obtain target overexposure feature information according to the first overexposure feature information and the second overexposure feature information; and obtain target underexposure feature information according to the first underexposure feature information and the second underexposure feature information; According to the feature information to be corrected, the target over-exposure feature information and the target under-exposure feature information are fused to obtain fused feature information; The target image is obtained according to the fused feature information and the feature image to be corrected.

[0007] Optionally, the above method performs feature mapping processing on the feature information to be corrected according to the overexposure mask information and the underexposure mask information to obtain the first overexposure feature information and the first underexposure feature information in the feature information to be corrected, including: Determining a feature image to be corrected and feature information to be corrected of the feature image to be corrected; Determine overexposure mask information and underexposure mask information according to the feature image to be corrected; According to the overexposure mask information, overexposure feature mapping processing is performed on the feature information to be corrected to obtain first overexposure feature information in the feature information to be corrected; According to the underexposure mask information, underexposure feature mapping processing is performed on the feature information to be corrected to obtain first underexposure feature information in the feature information to be corrected.

[0008] Optionally, the above method, according to the feature image to be corrected, determines the overexposure mask information and the underexposure mask information, including: Perform convolution activation processing on the feature image to be corrected to obtain an updated feature image to be corrected; Exposure classification processing is performed on the updated feature image to be corrected to obtain overexposure mask information and underexposure mask information of the updated feature image to be corrected.

[0009] Optionally, in the above method, the first overexposure feature information and the first underexposure feature information are respectively subjected to pooling convolution processing to obtain the second overexposure feature information and the second underexposure feature information, including: Determine the mask perception unit of the deexposure module in the target model; Inputting the first overexposure feature information into a pooling layer in a mask perception unit for pooling processing to obtain first target overexposure feature information in the first overexposure feature information; Inputting the first target overexposure feature information into the convolution layer in the mask perception unit for convolution processing to obtain the second target overexposure feature information in the first target overexposure feature information; Inputting the second target overexposure feature information into the activation layer in the mask perception unit for activation processing to obtain second overexposure feature information; Inputting the first underexposure feature information into a pooling layer in a mask perception unit for pooling processing to obtain first target underexposure feature information in the first underexposure feature information; Inputting the first target underexposure feature information into the convolution layer in the mask perception unit for convolution processing to obtain the second target underexposure feature information in the first target underexposure feature information; The underexposure feature information of the second target is input into the activation layer in the mask perception unit for activation processing to obtain the second underexposure feature information.

[0010] Optionally, in the above method, according to the feature information to be corrected, the target overexposure feature information and the target underexposure feature information are fused to obtain fused feature information, including: Performing feature channel updating processing on the feature information to be corrected and the target overexposure feature information to obtain first feature information; Performing feature channel updating processing on the feature information to be corrected and the target underexposure feature information to obtain second feature information; The first feature information and the second feature information are summed to obtain fused feature information.

[0011] Optionally, the above method, according to the fusion feature information and the feature image to be corrected, obtains the target image, including: Determine the refined modules in the target model; Inputting the fused feature information into the feature determination unit in the refining module to perform translation feature information and scaling feature information determination processing on the fused feature information to obtain translation feature information and scaling feature information; Inputting the scaled feature information into a feature fusion unit in a refinement module to perform convolution processing, activation processing, and model learning processing on the scaled feature information to obtain target fused feature information; The translation feature information, the scaling feature information, and the target fusion feature information are input into the image restoration unit in the refinement module to perform convolution processing on the translation feature information, the scaling feature information, and the target fusion feature information to obtain a target image.

[0012] Optionally, in the above method, the method for determining the target model further includes: Determine an initial model, first image information, and second image information, wherein the first image information is image information with abnormal exposure, and the second image information is image information with normal exposure corresponding to the first image information; Inputting the first image information into the initial model to obtain corrected image information; Comparing the corrected image information with the second image information for aesthetic features to obtain feature difference information; According to the feature difference information, the parameters of the initial model are adjusted to obtain the target model.

[0013] In a second aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; Memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method of the embodiment of the present application.

[0014] In a third aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method of the embodiment of the present application.

[0015] In a fourth aspect, the present application provides a computer program product, comprising a computer program, which implements any method of the first aspect when executed by a processor.

[0016] The estimated mixed exposure processing method, device, medium and product provided by the present application maps the mixed exposure image to the feature space to obtain the mixed exposure feature image, identifies the underexposed or overexposed regional images in the mixed exposure feature image, classifies and extracts each regional image, and processes the exposure information of each regional image respectively, so as to obtain each regional image with unified exposure. Finally, each regional image with unified exposure is effectively combined into a feature image after unified exposure according to the size and other information of the mixed exposure image, and restored to the image space to obtain the means of the target model. The underexposed part and the overexposed part in the exposure image can be identified at the same time, and the overexposed regional features and the underexposed regional features in the exposure image can be extracted and exposure correction can be performed respectively, thereby improving the efficiency of exposure correction and enhancing the user's viewing experience of the image. Summary of the invention

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] Figure 1 A schematic flow chart of a hybrid exposure processing method based on aesthetic evaluation provided in an embodiment of the present application; Figure 2 A schematic flow chart of another hybrid exposure processing method based on aesthetic evaluation provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a region-aware de-exposure module in a target model provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a refining module in a target model provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of the aesthetic evaluation module provided in the embodiment of the present application; Figure 6 A schematic structural diagram of a hybrid exposure processing device based on aesthetic evaluation provided in an embodiment of the present application; Figure 7 A schematic structural diagram of another hybrid exposure processing device based on aesthetic evaluation provided in an embodiment of the present application; Figure 8A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0019] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0020] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0021] Existing exposure correction methods only emphasize identifying and correcting a single scene of underexposure or overexposure. When encountering a scene with uneven lighting, a single image may be overexposed or underexposed. Therefore, when only one correction method is used, the correction process is too different and may not be able to simultaneously identify underexposed or overexposed images, causing the overexposed image to darken or the underexposed image to brighten, resulting in poor generalization ability for opposite exposure, which limits the deployment of exposure correction methods in practical applications.

[0022] In order to solve the above technical problems, the hybrid exposure processing method based on aesthetic evaluation provided in the present application proposes a hybrid exposure correction model. Through the exposure mask prediction module in the hybrid exposure correction model, the overexposed areas and underexposed areas in the hybrid exposure image are identified, and the overexposed areas and underexposed areas are mapped to the encoding space to obtain overexposure mask information and underexposure mask information. Then, the region-aware deexposure module in the hybrid exposure correction model is used to finely align the overexposed areas and underexposed areas through multi-branch mask-aware instance normalization to obtain an image with consistent exposure. At the same time, a model regularization strategy is also formulated to optimize the hybrid exposure correction model to obtain a preset exposure correction model, thereby ensuring uniform exposure distribution across regions, and achieving the goal of shortening the distance between the corrected image and the corresponding area of ​​the real image in the feature space to ensure exposure consistency within the region. At the same time, the exposure distance between the regions is modeled and compared to ensure exposure continuity, and finally a balanced improvement of the hybrid exposure image is achieved, solving the problem that the existing exposure correction technology cannot simultaneously identify and correct overexposed areas and underexposed areas.

[0023] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0024] The execution subject of the hybrid exposure processing method based on aesthetic evaluation provided in the embodiment of the present application may be a server, wherein the server may be a computer or other equipment. The embodiments of the present application do not impose any special restrictions on the implementation method of the execution subject, as long as the execution subject can perform feature mapping processing on the feature information to be corrected according to the overexposure mask information and the underexposure mask information, respectively, to obtain the first overexposure feature information and the first underexposure feature information in the feature information to be corrected, the overexposure mask information is obtained according to the overexposed area image in the feature image to be corrected, and the underexposure mask information is obtained according to the underexposed area image in the feature image to be corrected, and the feature information to be corrected is the feature information of the feature image to be corrected; the first overexposure feature information and the first underexposure feature information are respectively subjected to pooling convolution processing to obtain the second overexposure feature information and the second underexposure feature information; the target overexposure feature information is obtained according to the first overexposure feature information and the second overexposure feature information; and the target underexposure feature information is obtained according to the first underexposure feature information and the second underexposure feature information; according to the feature information to be corrected, the target overexposure feature information and the target underexposure feature information are fused to obtain fused feature information; and the target image is obtained according to the fused feature information and the feature image to be corrected.

[0025] The feature image to be corrected refers to a feature image of an unevenly exposed image in a feature space, and the unevenly exposed image refers to an exposed image that includes both underexposed areas and overexposed areas.

[0026] Overexposed areas refer to areas in the image where the brightness is too high during the photography or video recording process, causing these areas to lose image details and texture, appearing overexposed.

[0027] Underexposed areas refer to areas in the image where the brightness is too low during the photography or video recording process, causing these areas to lose image details and clarity, appearing underexposed.

[0028] Mask information can refer to a binary image, which is information used to identify overexposed or underexposed areas in an image in digital image processing. For example, overexposed areas can be marked in red, underexposed areas can be marked in green, and normally exposed areas can be marked in black. Through these mask information, users can quickly identify and locate overexposed and underexposed areas in the image for subsequent processing and adjustment. It can also be used in applications such as automatic exposure correction, dynamic range enhancement, and image restoration.

[0029] Figure 1 A hybrid exposure processing method based on aesthetic evaluation is provided in an embodiment of the present application. Figure 1 As shown, including: S101. According to the overexposure mask information and the underexposure mask information, feature mapping processing is performed on the feature information to be corrected respectively to obtain the first overexposure feature information and the first underexposure feature information in the feature information to be corrected. The overexposure mask information is obtained based on the overexposed area image in the feature image to be corrected, and the underexposure mask information is obtained based on the underexposed area image in the feature image to be corrected. The feature information to be corrected is the feature information of the feature image to be corrected.

[0030] Among them, the overexposure mask information and the underexposure mask information are used to identify overexposed and underexposed areas in the mixed exposure image, which are usually manifested as very bright or very dark parts with lost details. The overexposure mask information and the underexposure mask information are information obtained by pixel binary classification after convolution processing of the feature information to be corrected according to the convolution function, so as to further identify and adjust the exposure information of the mixed exposure image; the feature image to be corrected refers to the image in the feature space obtained after the mixed exposure image is processed by convolution activation; the mixed exposure image is composed of an overexposed image and an underexposed image; the feature mapping processing can refer to mapping the feature information to be corrected into a high-dimensional space, so that different categories of features in the feature information to be corrected can be more easily classified in the high-dimensional space, that is, the feature information to be corrected is placed in the high-dimensional space for classification through convolution and activation functions to obtain overexposure feature information and underexposure feature information.

[0031] Optionally, according to the overexposure mask information and the underexposure mask information, feature mapping processing is performed on the feature information to be corrected respectively to obtain first overexposure feature information and first underexposure feature information in the feature information to be corrected, including: Determining a feature image to be corrected and feature information to be corrected of the feature image to be corrected; Determine overexposure mask information and underexposure mask information according to the feature image to be corrected; According to the overexposure mask information, overexposure feature mapping processing is performed on the feature information to be corrected to obtain first overexposure feature information in the feature information to be corrected; According to the underexposure mask information, underexposure feature mapping processing is performed on the feature information to be corrected to obtain first underexposure feature information in the feature information to be corrected.

[0032] Among them, in order to extract and increase important features in the mixed exposure image to be corrected, so as to better perform image processing and correction, the present application first performs preset convolution and activation function operations on the mixed exposure image to be corrected, maps the mixed exposure image to be corrected into the feature space, and obtains the feature image to be corrected, so that the feature information in the image can be directly extracted for subsequent image recognition, classification and detection tasks; in addition, the feature space is usually lower in dimension than the original image space, which is convenient for subsequent reduction of calculation complexity while retaining important information in the image; the process of obtaining the feature image to be corrected includes resizing or normalizing the mixed exposure image to be corrected, and then inputting the mixed exposure image to be corrected into the convolutional neural network; selecting one or more convolution kernels, sliding the convolution kernel from the upper left corner of the mixed exposure image to be corrected pixel by pixel, and calculating the convolution kernel and the image The dot product sum of the local area; each sliding will get a value, and the feature map to be corrected of the mixed exposure image to be corrected is composed according to the various values; the feature map to be corrected is nonlinearly transformed by the activation function to introduce nonlinear features; according to the combined operation of convolution and activation function, the mixed exposure image to be corrected is mapped to a feature space, which can be a high-dimensional feature space, containing rich feature information of the mixed exposure image to be corrected; in addition, in order to ensure that the size of the feature map of the final output is consistent with that of the mixed exposure image to be corrected, an appropriate padding strategy can be used in the convolution operation, for example, using the same padding strategy can ensure that the output feature map is the same as the mixed exposure image to be corrected; after the above steps, the feature map to be corrected and the feature information of the feature map to be corrected can be obtained for subsequent image correction, enhancement or other processing operations.

[0033] Secondly, by updating the feature image to be corrected, the overexposure mask information and underexposure mask information in the image to be corrected can be obtained; by multiplying the feature image to be corrected with the overexposure mask information and the underexposure mask information respectively, the local area of ​​the feature image to be corrected can be identified and corrected without affecting other parts of the image. This local hybrid exposure processing method based on aesthetic evaluation can more accurately adjust the brightness and contrast of the image.

[0034] In addition, the purpose of multiplying the feature image to be corrected with the overexposure mask information and the underexposure mask information respectively, rather than multiplying it with the overexposure mask information and then continuing to multiply it with the underexposure mask information, is to independently correct the overexposed and underexposed areas in the image, so that different processing strategies can be adopted for different exposure problems to improve the accuracy and effect of correction; if the feature image to be corrected is first multiplied with the overexposure mask information and then multiplied with the underexposure mask information, the model pixels may be modified after the first multiplication, thereby affecting the result of the second multiplication, which may cause information loss or inaccurate correction. Therefore, separate multiplication can ensure that each mask information can act completely on the original image.

[0035] The calculation formulas for the first overexposure characteristic information and the first underexposure characteristic information are as follows:

[0036] in, Indicates the amount of overexposure. Indicates the amount of underexposure. Indicates overexposure mask information. represents underexposure mask information; through the above steps, the feature image to be corrected can be combined with the overexposure mask information and the underexposure mask information to obtain the overexposure component and the underexposure component of the feature image, thereby realizing the separation and processing of the underexposed and overexposed areas of the image.

[0037] Optionally, determining overexposure mask information and underexposure mask information according to the feature image to be corrected includes: Perform convolution activation processing on the feature image to be corrected to obtain an updated feature image to be corrected; Exposure classification processing is performed on the updated feature image to be corrected to obtain overexposure mask information and underexposure mask information of the updated feature image to be corrected.

[0038] Among them, the updated feature image to be corrected includes the feature information of the convolution kernel extracted for the feature information to be corrected, which may have different feature dimensions and includes the results of nonlinear transformation and local information aggregation. These changes enable the updated feature image to better represent the characteristics of the input data, thereby better performing feature processing and classification in subsequent neural network layers.

[0039] After determining the feature image to be corrected, a convolutional neural network is used to perform image processing on the feature image to be corrected, wherein the convolution layer in the convolutional neural network extracts the feature information in the feature image to be corrected, and the activation function performs nonlinear operations, thereby increasing the expression ability of the model, that is, the expression ability of the feature information, and then obtaining an updated feature image to be corrected; the updated feature image to be corrected is input into the Sigmoid function, and the Sigmoid function formula is as follows:

[0040] According to the above calculation method, the value of each pixel in the updated feature image to be corrected can be mapped to a range between 0 and 1, and pixels less than a certain threshold can be regarded as one category, and pixels greater than the threshold can be regarded as another category; for example, 0.5 can be used as a threshold, pixels less than 0.5 are classified as underexposed pixel information, and pixels greater than 0.5 are classified as overexposed pixel information. The above underexposed pixel information is the underexposure mask information, and the overexposed pixel information is the overexposure mask information.

[0041] S102: Perform pooling convolution processing on the first overexposure feature information and the first underexposure feature information respectively to obtain second overexposure feature information and second underexposure feature information.

[0042] Among them, in order to avoid the above-mentioned underexposure mask information or overexposure mask information being too rough to extract the regional features of the feature image to be corrected, it is necessary to perform pooling, convolution and activation processing on the first overexposure feature information and the first underexposure feature information so as to improve the accuracy and robustness of feature extraction, thereby more accurately extracting the regional features of the feature image to be corrected.

[0043] Optionally, performing pooling convolution processing on the first overexposure feature information and the first underexposure feature information respectively to obtain second overexposure feature information and second underexposure feature information includes: Determine the mask perception unit of the deexposure module in the target correction model; Inputting the first overexposure feature information into a pooling layer in a mask perception unit for pooling processing to obtain first target overexposure feature information in the first overexposure feature information; Inputting the first target overexposure feature information into the convolution layer in the mask perception unit for convolution processing to obtain the second target overexposure feature information in the first target overexposure feature information; Inputting the second target overexposure feature information into the activation layer in the mask perception unit for activation processing to obtain second overexposure feature information; Inputting the first underexposure feature information into a pooling layer in a mask perception unit for pooling processing to obtain first target underexposure feature information in the first underexposure feature information; Inputting the first target underexposure feature information into the convolution layer in the mask perception unit for convolution processing to obtain the second target underexposure feature information in the first target underexposure feature information; The underexposure feature information of the second target is input into the activation layer in the mask perception unit for activation processing to obtain the second underexposure feature information.

[0044] Among them, the present application proposes a target model that can uniformly de-exposure mixed exposure images with both overexposure and underexposure. The target model includes a mask perception module, and the mask perception unit includes a pooling processing unit, a convolution unit, and an activation unit; The pooling layer is used to reduce the size of the feature map, and by performing average pooling and maximum pooling processing on the first underexposed feature information and the first overexposed feature information, the spatial size of the overexposed feature region and the underexposed feature pre-fetch is reduced, and important feature information is retained, detail noise is ignored, the generalization ability of the target model is enhanced, and overfitting is reduced, thereby obtaining the first target overexposed feature information and the first target underexposed feature information; After obtaining the first target overexposure feature information and the first target underexposure feature information, the above feature information can be subjected to feature extraction and fusion processing again through the convolution layer, so as to retain more feature detail information while extracting important features. For example, by performing convolution processing on the first updated overexposure feature information and the first updated underexposure feature information through a three-by-three convolution kernel, feature extraction and information fusion can be performed through a small range of local perception, which is helpful to extract local patterns and details in the image, such as edges, corners, etc., so as to obtain the second target overexposure feature information and the second target underexposure feature information; Finally, the overexposed feature information of the second target and the underexposed feature information of the second target are processed by the Sigmoid function in the activation layer by introducing nonlinear transformation, limiting the value range, generating probability output and realizing smooth transition, so as to improve the expression ability and stability of the feature information, thereby obtaining the second overexposed feature information and the second underexposed feature information.

[0045] S103, obtaining target overexposure feature information according to the first overexposure feature information and the second overexposure feature information; and obtaining target underexposure feature information according to the first underexposure feature information and the second underexposure feature information.

[0046] Among them, after obtaining the first overexposure feature information, the second overexposure feature information, the first underexposure feature information, and the second underexposure feature information, in order to improve the effect of image processing and analysis, it is necessary to perform instance normalization processing on the first overexposure feature information and the second overexposure feature information, and to perform instance normalization processing on the first underexposure feature information and the second underexposure feature information; wherein, through normalization processing, the change in the distribution of input data of each layer during the training of the neural network can be reduced, which helps to improve the stability and speed of training; it can improve the contrast of the feature map, making the overexposure feature more obvious, which is helpful for subsequent image processing and analysis; by unifying the data distribution of different feature maps to the same scale, it helps to reduce the range difference of the feature value, making the features easier to fuse and compare; in addition, instance normalization helps the model to better learn the true distribution of the data, thereby improving the performance of the model on the test set and enhancing the generalization ability. After normalization, the first overexposure feature information and the second overexposure feature information are converted to target overexposure feature information of a relatively uniform scale, and the first underexposure feature information and the second underexposure feature information are converted to target underexposure feature information of a relatively uniform scale, so that subsequent convolution, feature fusion and other operations can be performed more efficiently, ultimately improving the performance of the entire image processing system. The formula for normalization is as follows:

[0047]

[0048] in, For overexposure, For underexposure, represents the overexposed component feature after maximum pooling, represents the overexposed component characteristics after average pooling, represents the underexposed component feature after maximum pooling, represents the underexposed component characteristics after average pooling, Represents the sigmoid function.

[0049] S104 , fusing the target overexposure feature information and the target underexposure feature information according to the feature information to be corrected to obtain fused feature information.

[0050] Among them, after obtaining the target overexposure feature information and the target underexposure feature information after layer-by-layer optimization processing, it is necessary to fuse the target overexposure feature information and the target underexposure feature information to obtain feature image information with unified exposure, wherein unified exposure means that the final generated feature image has achieved balance and consistency in brightness, contrast and color.

[0051] Optionally, according to the feature information to be corrected, the target overexposure feature information and the target underexposure feature information are fused to obtain fused feature information, including: Performing feature channel updating processing on the feature information to be corrected and the target overexposure feature information to obtain first feature information; Performing feature channel updating processing on the feature information to be corrected and the target underexposure feature information to obtain second feature information; The first feature information and the second feature information are summed to obtain fused feature information.

[0052] Among them, the feature channel update processing refers to the unified processing of the feature channels of the feature information to be corrected and the target overexposed feature information to obtain information with the same number of channels. For example, through convolution processing, the feature image to be corrected and the target overexposed feature image or the target underexposed feature image can be fused together, so that the target model can simultaneously consider the features of different levels in the feature image. Among them, the convolution processing with a convolution kernel of 1×1 can be used to change the number of channels of the feature map without changing the spatial dimension of the feature map. That is, if the number of channels of the feature image to be corrected and the processed exposure feature map are different, their number of channels can be unified through convolution, which is convenient for subsequent processing; in addition, combined with the activation function Sigmoid, nonlinear transformation can be introduced to help the target model learn more complex feature relationships in the feature image; finally, the feature information of the overexposed part and the underexposed part after convolution processing are combined together to obtain the target fused feature information with the same space and dimension as the feature image to be corrected. The convolution processing formula is as follows:

[0053] in, is the feature image information to be corrected, is the target overexposure feature information, It is the target overexposure characteristic information.

[0054] In addition, in the target model, after obtaining the target fusion feature information, the target fusion feature information can be replaced with the feature image to be corrected, and the corrected feature image can be subjected to average pooling, maximum pooling, convolution, activation, etc. again. Through multiple rounds of processing, the target model can extract and process higher-level feature information layer by layer, and the feature map after each round of processing can capture richer and more complex features, thereby improving the model's expression and recognition capabilities; secondly, multiple rounds of processing can gradually refine and optimize the features. After each round of processing, the model can refine the features, making important features more prominent, and suppressing noise and redundant information. Such a process can enhance the model's sensitivity and recognition capabilities to target features; in addition, through multiple rounds of processing, features at different levels can be fused and interacted multiple times, thereby enhancing the effect of feature fusion. For example, after the initial feature information and the processed exposure feature information are fused in the first convolution process, further processing can further fuse more contextual information and local information; multiple rounds of processing can hierarchical feature representation, making the feature representation of each layer more abstract and high-order, which is particularly important for processing complex visual tasks.

[0055] S105: Obtain a target image based on the fused feature information and the feature image to be corrected.

[0056] Optionally, obtaining a target image according to the fused feature information and the feature image to be corrected includes: Determine the refined modules in the target correction model; Inputting the fused feature information into the feature determination unit in the refining module to perform translation feature information and scaling feature information determination processing on the fused feature information to obtain translation feature information and scaling feature information; Inputting the scaled feature information into a feature fusion unit in a refinement module to perform convolution processing, activation processing, and model learning processing on the scaled feature information to obtain target fused feature information; The translation feature information, the scaling feature information, and the target fusion feature information are input into the image restoration unit in the refinement module to perform convolution processing on the translation feature information, the scaling feature information, and the target fusion feature information to obtain a target image.

[0057] Among them, by integrating various fusion feature information, the feature image can be processed uniformly, and the integration process includes: calculation of translation coefficient component: by convolution processing the fused feature information, the translation coefficient component is obtained, and a translation coefficient can be generated to adjust the position or baseline of the feature information; after an activation function, it is processed by a layer of convolution; then a scaling coefficient component is learned by the average pooling layer to generate a scaling coefficient to adjust the amplitude or size of the feature information; after an activation function, it is processed by a layer of convolution; finally, a new fusion feature is obtained by the Sigmoid activation function; wherein, the feature information is adjusted based on the scaling coefficient, and is normalized to the range of 0 to 1 by the Sigmoid function, so that the feature information is more stable and meaningful. For channel restoration, a layer of convolution is used to process the new fusion feature, and its channel number is restored from the feature space to the image space. The processed feature information can be converted back to a form suitable for subsequent image processing or analysis, so that it can be further used or decoded. Among them, the convolution processing can be a convolution with a convolution kernel of three times three, and the calculation formula is as follows:

[0058] in, The above process can uniformly process feature information from multiple sources or at different levels to make the information more complete and representative; the feature information is appropriately adjusted through translation and scaling coefficients to improve the expressiveness and robustness of the features; at the same time, nonlinear transformations are introduced through activation functions to enable the model to better learn complex feature patterns; the feature information is normalized to a specific range through activation functions such as Sigmoid functions; finally, the processed feature information is converted back to a form suitable for the image space to obtain the target image, so that it can be used for subsequent image processing tasks to improve the quality of image expression.

[0059] Optionally, the process of determining the target model includes: Determine an initial model, first image information, and second image information, wherein the first image information is image information with abnormal exposure, and the second image information is image information with normal exposure corresponding to the first image information; Inputting the first image information into the initial model to obtain corrected image information; Comparing aesthetic features between the corrected image information and the second image information to obtain feature difference information; According to the feature difference information, the parameters of the initial model are adjusted to obtain the target model.

[0060] Among them, the target model proposed in this application can perform unified de-exposure processing on mixed exposure images with both overexposure and underexposure phenomena. The determination process of the target model is as follows: An initial model for processing abnormal exposure image information is determined, wherein the initial model includes a correction algorithm and a learning algorithm for correcting abnormal exposure images; first, abnormal exposure image information that needs to be exposed and normal image information that does not need to be exposed is obtained; the abnormal exposure image information is input into the initial model, and the exposure prediction image is corrected by the correction algorithm in the initial model to obtain corrected image information; the corrected image information is compared with the normal exposure image information for aesthetic features, and feature difference information such as L1 distance, cosine distance, binary cross entropy loss, etc. between the corrected image information and the normal exposure image information can be obtained; based on the above-mentioned feature difference information such as L1 distance, cosine distance, binary cross entropy loss, etc., an exposure contrast regularization loss is obtained; based on the exposure contrast regularization loss and the regularization loss of the aesthetic evaluation prior, an initial hybrid exposure correction model is adjusted so that the learning algorithm in the initial model adjusts the parameters of the initial model based on the exposure contrast regularization loss and the regularization loss of the aesthetic evaluation prior, so that the correction model can adapt to different types of abnormal exposure conditions, thereby obtaining a target model.

[0061] The hybrid exposure processing method based on aesthetic evaluation provided in the embodiment of the present application can perform feature mapping processing on the feature information to be corrected according to the overexposure mask information and the underexposure mask information, respectively, to obtain the first overexposure feature information and the first underexposure feature information in the feature information to be corrected, the overexposure mask information is obtained according to the overexposed area image in the feature image to be corrected, the underexposure mask information is obtained according to the underexposed area image in the feature image to be corrected, and the feature information to be corrected is the feature information of the feature image to be corrected; the first overexposure feature information and the first underexposure feature information are respectively subjected to pooling convolution processing to obtain the second overexposure feature information and the second underexposure feature information; the target overexposure feature information is obtained according to the first overexposure feature information and the second overexposure feature information; and the target underexposure feature information is obtained according to the first underexposure feature information and the second underexposure feature information; according to the feature information to be corrected, the target overexposure feature information and the target underexposure feature information are fused to obtain the fused feature information; the target image is obtained according to the fused feature information and the feature image to be corrected, so as to effectively improve and process the image with uneven exposure, enhance the underexposed area and suppress the overexposed area, thereby improving the efficiency of identifying and processing the hybrid exposure image.

[0062] Figure 2 Another hybrid exposure processing method based on aesthetic evaluation is provided in the embodiment of the present application. Figure 2As shown, based on the above embodiment, the method for determining the target model, the process of the hybrid exposure processing method based on aesthetic evaluation may include: S201: Determine an initial model for hybrid exposure correction.

[0063] Among them, firstly, a suitable deep learning model structure is selected for hybrid exposure correction, and the input and output structure of the model is defined to ensure that the model can receive poorly exposed images and generate enhanced images; the hybrid exposure correction model is initialized, including an initialized region-aware de-exposure module, an initialized exposure mask prediction module, an initialized refinement module, and a pre-trained and parameter-frozen aesthetic evaluation module; The region-aware deexposure module consists of a three-branch instance normalization layer and a mask instance normalization layer. Each time an image passes through a layer, a feature image of a specific size is obtained. The width and height dimensions are the same as the input, but the number of channels is increased, which contains the high-level features of the original image. The exposure mask prediction module consists of a series of convolutional layers and activation layers. Since the predicted mask requires binary classification of pixels, a Sigmoid layer is added at the end. The refinement module consists of a residual channel attention block. Since it is necessary to learn multi-channel feature maps, it is necessary to strengthen important channels and suppress unimportant channels as much as possible. Therefore, the residual channel attention block is mainly composed of two branches, one of which is used to learn the weight of each channel, and the other is used to learn the feature map of each channel. The processing process of the refinement module is as follows: Receive the target feature information sent by the mask perception module, then perform a three-by-three convolution on the target feature information, slide a three-by-three convolution kernel on the feature map to extract local spatial information, the size of the convolution kernel is three-by-three, which means that a weighted sum operation within a three-by-three range is performed at each position to obtain a new feature map; then, the convolution result is passed through the Sigmoid activation function, and the Sigmoid function maps the input value to between 0 and 1, so that the eigenvalue is normalized and non-linear features are added; finally, the activated feature map is subjected to an average pooling operation, which reduces the size of the feature map by taking the average value in a certain area, while retaining the main feature information and reducing overfitting, and finally obtaining the initial feature fusion information; the initial feature fusion information is subjected to the first three-by-three convolution to further extract the feature information and obtain the convolution result, and then the Leaky activation function is used to process the convolution result, and the Leaky activation function is used to process the convolution result. The ReLU activation function can solve the neuron problem of the standard ReLU when negative value input occurs. Then, the activated feature map is subjected to three-by-three convolution again to further extract and fuse features. The convolution result is passed through the Sigmoid activation function to normalize the feature value again, thereby obtaining the target feature fusion information. At this time, the target feature fusion information is obtained through multiple convolution and activation processes, and contains richer and more detailed feature information. Finally, the target feature fusion information is subjected to three-by-three convolution to convert the target fusion feature information into the final image information to obtain the target image. The formula is as follows:

[0064]

[0065] in, is the translation coefficient component, is the scaling factor component, Fusion information for target features.

[0066] The aesthetic evaluation module includes an aesthetic evaluation unit, in which an aesthetic evaluation prior regularization (AEPR) method for image aesthetic evaluation is deployed, aiming to improve the computer vision system's ability to evaluate the aesthetic quality of images. Prior regularization is a method of introducing prior knowledge into a machine learning model to improve model performance. The aesthetic evaluation module learns the aesthetic features of the current environment through a large amount of data, and then keeps the parameters frozen during the application phase to ensure the consistency and stability of the evaluation. The aesthetic evaluation module can help users quickly and accurately evaluate the aesthetic features of images, such as composition, color matching, contrast, etc. The backbone network of the aesthetic evaluation module is mainly composed of the MobileNetV2 architecture, and ends with a fully connected prediction structure of ReLU-Linear-Sigmoid, which can score the quality of the generated-true value images, judge and supervise whether they meet human aesthetic standards, and finally obtain more harmonious high-quality generated images.

[0067] S202: Acquire a pair of images with abnormal exposure and images with normal exposure.

[0068] First, we define images with abnormal exposure and images with normal exposure. Abnormal exposure refers to the presence of overly bright or dark areas in the image, and normal exposure means that the exposure of the image should be within the appropriate brightness range, without overexposure or overbrightness. Pairs of poorly exposed images and aligned normally exposed images in natural scenes are collected as training samples, where it is necessary to ensure that the scenes and contents corresponding to the images are similar so that the correction rules for poorly exposed images can be learned when training the model. The collected image data are preprocessed, including image resizing, cropping, denoising and other operations, to ensure that the data format and quality of the input model are consistent.

[0069] S203: Input the image with abnormal exposure into the initial model to generate an enhanced image.

[0070] Among them, the above-obtained pairs of images with abnormal exposure and images with normal exposure are sorted into an abnormal exposure image dataset and a normal exposure image dataset; the data in the normal exposure image dataset and the abnormal exposure image dataset are preprocessed respectively, such as resizing, normalization, etc., so that they meet the model input format of the hybrid exposure correction model; the preprocessed abnormal exposure image dataset and normal exposure image dataset are input into the hybrid exposure correction model to complete the repair of the abnormal exposure image and improve the image quality, thereby generating an enhanced image.

[0071] S204: Generate a loss function according to the enhanced image and the normally exposed image.

[0072] First, the L1 distance loss between the enhanced image and the normally exposed image is calculated. The L1 distance loss is used to evaluate the difference between the model predicted image and the normally exposed image. The L1 distance loss can be expressed as:

[0073] Among them, pred represents the model output image, gt represents the paired image with normal exposure, which is used as the true value to supervise the model; the purpose of using L1 distance loss is that when the calculation goal is to regress the image at the pixel level, the use of L1 loss can reduce the sensitivity to outliers, thereby improving the robustness of the target model.

[0074] Secondly, the cosine distance loss between the enhanced image and the image with normal exposure is calculated to measure the difference between the exposure-corrected image and the normal exposure image. It is usually used for vector comparison in high-dimensional space. It is based on the cosine value of the angle between the two vectors, and the value range is 0 to 1, where 0 means exactly the same and 1 means completely different. In the exposure correction task, the cosine distance loss can effectively measure the similarity of the two images in the feature space. Compared with directly using pixel-level differences, such as mean square error, the cosine distance pays more attention to the direction and shape of the overall features of the image, rather than the absolute value. In addition, the cosine distance is insensitive to changes in the brightness of the image and mainly focuses on the direction of the vector. By minimizing the cosine distance loss, the model can be trained to pay more attention to the global feature consistency of the image, thereby improving the effect of exposure correction, so that the corrected image is not only closer to the normal exposure image in brightness, but also more consistent in overall structure and content. The cosine distance loss formula is as follows:

[0075] Then, the binary cross entropy distance loss function is used to measure the difference between the probability distribution predicted by the model and the true label; among them, the binary cross entropy loss is mainly used for binary classification tasks, but in the exposure correction task, if the problem is converted into a binary classification problem, that is, to determine whether each pixel is correctly exposed, the binary cross entropy loss can also play a role in model correction, mainly for the brightness space Y in the YCbCr color space, where YCbCr is a color space used for digital video and image processing. It decomposes color information into brightness Y and chrominance Cb, Cr components. The main advantage of the YCbCr color space is that it can effectively separate brightness and chrominance information, thereby better compressing and processing image and video data. The larger the value of the Y component, the brighter the image, and the smaller the value, the lower the brightness; For the binary cross entropy distance loss, we first need to calculate the true value of the exposure mask M gt , the calculation process can be expressed as:

[0076] in, Y i Describing the i-th pixel value on the Y channel of the image, the binary cross entropy loss can be expressed as:

[0077] Where N represents the mask length, which is the total number of pixels in the image.

[0078] In addition, in order to prevent the final target model from overfitting and improve the generalization ability of the model, so that it performs more stably and robustly when processing images under different exposure conditions, the present application also proposes a regularization function to train the exposure correction model, that is, when training the exposure correction model, if the model is too complex or has too many parameters, it is easy to overfit the noise and specific patterns in the training data, so that the performance on the new data deteriorates. Regularization limits the complexity of the model and reduces overfitting by optimizing the parameters of the model; secondly, regularization can make the parameter distribution of the model smoother and more uniform, avoiding some parameters that are too large or too small, so that the model is smoother and more robust to input changes; in addition, by adding regularization terms during the training process, the generalization ability of the model can be effectively improved, so that the model performs better when processing unseen data, that is, under different exposure conditions. The specific regularization process is as follows: The generated image is divided into overexposed areas using the exposure mask prediction module and underexposed areas , the regularization function is as follows:

[0079] Among them, the function D(·) represents the L1 distance function, They represent the feature vectors generated from the pre-trained VGG16, and their inputs are the generated image, the true image, and the overexposed area of ​​the input image. and underexposed areas , parameter c represents the correlation between overexposure and underexposure styles, and is calculated as:

[0080] Aesthetic evaluation prior regularization mainly considers the difference in aesthetic evaluation scores between generated images and true images:

[0081] in, represents the pre-trained and parameter-frozen aesthetic evaluation module, Represents the hybrid exposure correction model; VGG16 is a deep convolutional neural network that is widely used in tasks such as image classification and feature extraction. In the aesthetic evaluation task, VGG16 can be used as a feature extractor to extract high-level features of the image, and then perform aesthetic scoring based on these features. The specific process is as follows: Determine a pre-trained VGG16 model and use the pre-trained VGG16 model to extract image features; use the VGG16 model to extract feature vectors of generated images and true images; calculate aesthetic scores, and use an aesthetic evaluation model to calculate the aesthetic scores of generated images and true images based on the extracted feature vectors. The aesthetic evaluation model can be a simple fully connected network or other regression model; calculate the difference in aesthetic scores between the generated image and the true image; introduce an a priori regularization function to penalize large score differences, and then obtain the difference in aesthetic evaluation scores between the generated image and the true image.

[0082] S205. According to the loss function, the initial model is trained and iterated a preset number of times to obtain a target model.

[0083] Among them, select a suitable optimizer to minimize the loss function. Common optimizers include stochastic gradient descent, etc.; in each training iteration, input the training data into the mixed exposure correction model, calculate the loss function, and update the model parameters through the back propagation algorithm; repeat the above steps until the preset number of training times is reached; after the training is completed, use the validation set or evaluation set to evaluate the performance of the model and check whether the model can effectively verify the mixed exposure problem; save the trained preset mixed exposure model for subsequent use in practical applications.

[0084] The method provided in the embodiment of the present application can perform training iterations on the mixed exposure correction model for a preset number of times to obtain a preset mixed exposure model, which can be used to automatically correct the mixed exposure problem in the image.

[0085] Figure 3 This is a schematic diagram of the structure of the region-aware de-exposure module in the target model provided in the embodiment of the present application. Figure 3 As shown, the region-aware de-exposure module 30 includes a feature mapping unit 301, a mask perception unit 302, an instance normalization unit 303, and a fusion unit 304, wherein: A feature mapping unit 301 is used to extract first overexposure feature information and first underexposure feature information of a feature image to be corrected; The mask sensing unit 302 is used to perform convolution pooling processing on the first overexposure feature information and the first underexposure feature information, and output target overexposure feature information and target underexposure feature information; The instance normalization unit 303 is used to perform instance normalization processing on the feature information to be corrected, the target overexposure feature information, and the target underexposure feature information, and output overexposure feature normalization information and underexposure feature normalization information; The fusion unit 304 is used to fuse the overexposure feature normalization information and the underexposure feature normalization information, and output fused feature information.

[0086] Figure 4 This is a schematic diagram of the structure of the refinement module in the target model provided in the embodiment of the present application. Figure 4 As shown, the refining module 40 includes a feature determination unit 401, a feature fusion unit 402, and an image restoration unit 403, wherein: The feature determination unit 401 is used to perform a convolution process with a convolution kernel of three times three on the fused feature information output by the area-aware de-exposure module 30, and output a translation coefficient component; perform a convolution process with a convolution kernel of three times three, an activation process, a convolution process with a convolution kernel of three times three, and an average pooling process on the translation coefficient component to obtain a scaling coefficient component; A feature fusion unit 402 is used to perform convolution processing with a convolution kernel of three times three, activation processing, convolution processing with a convolution kernel of three times three, and sigmoid learning processing on the scaling coefficient classification to obtain target fusion feature information; The image restoration unit 403 is used to perform convolution processing with a convolution kernel of three times three on the translation coefficient component, the scaling coefficient component, and the target fusion feature information to obtain a target image.

[0087] Figure 5 This is a schematic diagram of the structure of the aesthetic evaluation module provided in the embodiment of the present application. Figure 5 As shown, the aesthetic evaluation module 50 includes a convolution processing unit 501 and a feature optimization unit 502, wherein: A convolution processing unit 501 is used to perform convolution processing on the target image to obtain target aesthetic feature information; The feature optimization unit 502 is used to perform learning processing on the target aesthetic feature information to obtain aesthetic evaluation information of the target image.

[0088] Figure 6 This is a schematic diagram of the structure of a hybrid exposure processing device based on aesthetic evaluation provided in an embodiment of the present application. Figure 6 As shown, the hybrid exposure processing device 60 based on aesthetic evaluation includes an exposure mask prediction module 601, a region-aware de-exposure module 602, and a refinement module 603, wherein: The exposure mask prediction module 601 is used to map the image to be corrected to the coding space, obtain the feature image to be corrected of the same size as the feature image to be corrected, and the overexposure mask information and underexposure mask information of the feature image to be corrected; The region-aware de-exposure module 602 is used to perform unified exposure processing on the over-exposure mask information and the under-exposure mask information of the feature image to be corrected to obtain a unified exposure feature image; The refining module 603 is used to restore the exposure unified feature image from the feature space to the image space to obtain the target image.

[0089] Figure 7 This is a schematic diagram of the structure of another hybrid exposure processing device based on aesthetic evaluation provided in an embodiment of the present application. Figure 4 As shown, the hybrid exposure processing device 70 based on aesthetic evaluation includes a first feature information obtaining module 701, a second feature information obtaining module 702, a target feature information obtaining module 703, a fusion feature information obtaining module 704, and a target image obtaining module 705, wherein: The first feature information obtaining module 701 is used to perform feature mapping processing on the feature information to be corrected according to the overexposure mask information and the underexposure mask information, respectively, to obtain the first overexposure feature information and the first underexposure feature information in the feature information to be corrected. The overexposure mask information is obtained based on the overexposed area image in the feature image to be corrected, and the underexposure mask information is obtained based on the underexposed area image in the feature image to be corrected. The feature information to be corrected is the feature information of the feature image to be corrected.

[0090] The second feature information obtaining module 702 is used to perform pooling convolution processing on the first overexposure feature information and the first underexposure feature information respectively to obtain second overexposure feature information and second underexposure feature information.

[0091] The target characteristic information obtaining module 703 is used to obtain target overexposure characteristic information according to the first overexposure characteristic information and the second overexposure characteristic information; and to obtain target underexposure characteristic information according to the first underexposure characteristic information and the second underexposure characteristic information.

[0092] The fused feature information obtaining module 704 is used to fuse the target overexposure feature information and the target underexposure feature information according to the feature information to be corrected, so as to obtain fused feature information.

[0093] The target image obtaining module 705 is used to obtain the target image according to the fused feature information and the feature image to be corrected.

[0094] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 80 includes: The electronic device 80 may include a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a communication component 803 and other components. The processor 801 , the memory 802 and the communication component 803 are connected via a bus 804 .

[0095] In a specific implementation process, at least one processor 801 executes the computer-executable instructions stored in the memory 802, so that at least one processor 801 executes the above hybrid exposure processing method based on aesthetic evaluation.

[0096] The specific implementation process of the processor 801 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0097] In the above Figure 8 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0098] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0099] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0100] In some embodiments, a computer program product is also proposed, including a computer program or instructions, which, when executed by a processor, implements the steps in any of the above-mentioned hybrid exposure processing methods based on aesthetic evaluation.

[0101] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0102] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0103] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple instructions are stored, and the instructions can be loaded by a processor to execute the steps of any one of the hybrid exposure processing methods based on aesthetic evaluation provided in the embodiments of the present application.

[0104] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0105] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program comprises computer instructions stored in a computer-readable storage medium.

[0106] Since the instructions stored in the storage medium can execute the steps of any one of the hybrid exposure processing methods based on aesthetic evaluation provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the hybrid exposure processing methods based on aesthetic evaluation provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0107] This embodiment also provides a computer program product, including a computer program, which, when executed by a processor, implements the hybrid exposure processing method based on aesthetic evaluation provided by any one of the above embodiments.

[0108] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0109] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0110] It should be further noted that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flow chart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0111] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0112] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0113] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. Unless otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc.

[0114] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0115] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0116] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A hybrid exposure processing method based on aesthetic evaluation, characterized in that: include: According to the overexposure mask information and the underexposure mask information, feature mapping processing is performed on the feature information to be corrected respectively to obtain first overexposure feature information and first underexposure feature information in the feature information to be corrected, wherein the overexposure mask information is obtained according to the overexposed region image in the feature image to be corrected, and the underexposure mask information is obtained according to the underexposed region image in the feature image to be corrected, and the feature information to be corrected is feature information of the feature image to be corrected; Performing pooling convolution processing on the first overexposure feature information and the first underexposure feature information respectively to obtain second overexposure feature information and second underexposure feature information; Obtain target overexposure feature information according to the first overexposure feature information and the second overexposure feature information; and obtain target underexposure feature information according to the first underexposure feature information and the second underexposure feature information; According to the feature information to be corrected, the target overexposure feature information and the target underexposure feature information are fused to obtain fused feature information; A target image is obtained according to the fused feature information and the feature image to be corrected.

2. The method according to claim 1, characterized in that The method of performing feature mapping processing on the feature information to be corrected according to the overexposure mask information and the underexposure mask information to obtain first overexposure feature information and first underexposure feature information in the feature information to be corrected includes: Determining a feature image to be corrected and feature information to be corrected of the feature image to be corrected; Determining overexposure mask information and underexposure mask information according to the characteristic image to be corrected; According to the overexposure mask information, performing overexposure feature mapping processing on the feature information to be corrected to obtain first overexposure feature information in the feature information to be corrected; According to the underexposure mask information, underexposure feature mapping processing is performed on the feature information to be corrected to obtain first underexposure feature information in the feature information to be corrected.

3. The method according to claim 2, characterized in that The determining of overexposure mask information and underexposure mask information according to the characteristic image to be corrected includes: Performing convolution activation processing on the feature image to be corrected to obtain an updated feature image to be corrected; Exposure classification processing is performed on the updated feature image to be corrected to obtain overexposure mask information and underexposure mask information of the updated feature image to be corrected.

4. The method according to claim 1, characterized in that: The performing pooling convolution processing on the first overexposure feature information and the first underexposure feature information respectively to obtain second overexposure feature information and second underexposure feature information includes: determining mask-aware units in a target model; Inputting the first overexposure feature information into a pooling layer in a mask perception unit for pooling processing to obtain first target overexposure feature information in the first overexposure feature information; Inputting the first target overexposure feature information into the convolution layer in the mask perception unit for convolution processing to obtain second target overexposure feature information in the first target overexposure feature information; Inputting the second target overexposure feature information into the activation layer in the mask sensing unit for activation processing to obtain second overexposure feature information; Inputting the first underexposure feature information into a pooling layer in the mask perception unit for pooling processing to obtain first target underexposure feature information in the first underexposure feature information; Inputting the first target underexposure feature information into the convolution layer in the mask perception unit for convolution processing to obtain second target underexposure feature information in the first target underexposure feature information; The second target underexposure feature information is input into the activation layer in the mask perception unit for activation processing to obtain the second underexposure feature information.

5. The method according to claim 1, characterized in that The step of fusing the target overexposure feature information and the target underexposure feature information according to the feature information to be corrected to obtain fused feature information includes: Performing feature channel updating processing on the feature information to be corrected and the target overexposure feature information to obtain first feature information; Performing feature channel updating processing on the feature information to be corrected and the target underexposure feature information to obtain second feature information; The first feature information and the second feature information are summed to obtain fused feature information.

6. The method according to claim 1, characterized in that The step of obtaining a target image according to the fused feature information and the feature image to be corrected comprises: Determine the refined modules in the target model; Inputting the fused feature information into the feature determination unit in the refining module to perform translation feature information and scaling feature information determination processing on the fused feature information to obtain translation feature information and scaling feature information; Inputting the scaled feature information into the feature fusion unit in the refining module to perform convolution processing, activation processing, and model learning processing on the scaled feature information to obtain target fused feature information; The translation feature information, the scaling feature information, and the target fusion feature information are input into the image restoration unit in the refinement module to perform convolution processing on the translation feature information, the scaling feature information, and the target fusion feature information to obtain a target image.

7. The method according to any one of claims 4 or 6, characterized in that include: Determine an initial model, first image information, and second image information, wherein the first image information is image information with abnormal exposure, and the second image information is image information with normal exposure corresponding to the first image information; Inputting the first image information into the initial model to obtain corrected image information; Comparing the aesthetic features of the corrected image information with the second image information to obtain feature difference information; According to the feature difference information, the parameters of the initial model are adjusted to obtain a target model.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images