Image mirror highlight removal method based on weakly supervised learning
By employing a weakly supervised learning-based method for removing specular highlights from images, and utilizing sparse nonnegative matrix factorization and recurrent generative adversarial networks, the method solves the color distortion problem of traditional methods, achieves efficient highlight removal, and simplifies the operation process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHONGSHAN FLASHLIGHT POLYTECHNIC
- Filing Date
- 2022-07-18
- Publication Date
- 2026-07-07
AI Technical Summary
Traditional highlight removal methods suffer from severe color distortion when processing natural images, especially when white areas are present. Deep learning-based methods, on the other hand, require a lot of additional information and are complex to operate.
We employ a weakly supervised learning-based method for removing specular highlights from images. Training data is obtained through sparse non-negative matrix factorization, and an end-to-end recurrent generative adversarial network architecture is used, including highlight generation, removal, and reconstruction modules. Multiple loss functions are utilized to optimize the removal of highlights from images.
It enables training using only highlight images, is simple to operate, and has good highlight removal effect. Compared with traditional and existing methods, it has wider applicability and better highlight removal effect.
Smart Images
Figure CN115170427B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image specular highlight removal, and more particularly to an image specular highlight removal method based on weakly supervised learning. Background Technology
[0002] Traditional specular removal methods (such as Tan, Akashi, and Yamamoto) perform poorly when processing natural images, resulting in severe color distortion, especially when white areas are present. Deep learning-based specular removal methods, such as Shi and Fu, perform well in image processing, but they require a large amount of additional information for training and are quite complex to implement. Summary of the Invention
[0003] To address the technical problem of severe distortion when traditional specular highlight removal methods process natural images, the present invention aims to provide an image specular highlight removal method based on weakly supervised learning.
[0004] This invention is achieved through the following technical solution:
[0005] The image specular highlight removal method based on weakly supervised learning includes the following steps:
[0006] First, the highlight region of the image is decomposed using a sparse non-negative matrix, and the image without highlight is cropped from the non-highlight region as training data.
[0007] Next, the training data is input into three joint training modules that are connected end-to-end to perform highlight generation, highlight removal and image reconstruction tasks respectively, and the network is trained using a recurrent generative adversarial network architecture to generate the final highlight-free image.
[0008] Furthermore, the step of decomposing the highlight region of the image using a sparse nonnegative matrix and cropping the non-highlight image from the non-highlight region as training data includes the following steps:
[0009] First, create the following collection:
[0010]
[0011] Next, the internal dimension of the matrix decomposition is set to 2, and k is solved by minimizing the l1 norm while maintaining the l2 norm. s (x), obtain the specular mask corresponding to the original image, use the specular mask to crop out the original specular image, and put the specular mask into the specular mask set. Randomly select a mask that does not overlap with the specular mask in the specular mask set to obtain the original non-spectral image.
[0012] In the formula, the left side of the equation represents the original image with a total number of pixels of n, and the matrix size is 3xn. The 3x2 matrix in the left half of the right side of the equation represents the fixed parameters of diffuse reflection and specular reflection on the RGB channels, and the right half is a 2xn matrix containing the diffuse reflection and specular reflection coefficients.
[0013] Furthermore, the step of inputting training data into three jointly trained modules connected end-to-end to perform highlight generation, highlight removal, and image reconstruction tasks respectively, and training the network using a recurrent generative adversarial network architecture to generate the final highlight-free image includes the following steps:
[0014] First, the highlight generation module converts the input original image without highlights into a transitional highlight image, and uses a highlight discrimination network to determine the authenticity of the transitional highlight image;
[0015] Next, the highlight removal module inputs the transition highlight image and removes its highlight portion to obtain the corresponding transition no-highlight image. The highlight removal discrimination network is then used to determine the authenticity of the transition no-highlight image.
[0016] Finally, the reconstruction module takes the original image, the transitional no-highlight image, and the highlight mask as input, and uses a pixel loss function to obtain the final no-highlight image.
[0017] Furthermore, the identity loss function is applied to optimize the transitional highlight image. The mathematical expression of the identity loss function is as follows:
[0018]
[0019] In the formula, G represents the specular generation module, E represents the expected value, p represents the data distribution, and I represents the specular value. h Represents the original specular image, ||G(I h ),I h ||1 represents the l1 norm.
[0020] Furthermore, the specular discrimination network uses the PatchGAN architecture to determine the authenticity of each 4x4 block in the transition specular image, and finally the average of all calculation results is taken.
[0021] The specular generation module and the specular discrimination network are optimized based on the objective function of PatchGAN, and the mathematical expression of the objective function is as follows:
[0022]
[0023] In the formula, G represents the specular generation module, and D... h This represents a specular discrimination network, where E represents the expectation, p represents the data distribution, and I represents the specular discrimination network. f Indicates an image without highlights, I hIndicates the original highlight image, I h ~p(I h This indicates that the image was selected from the original specular image data distribution.
[0024] Furthermore, the highlight generation module and the highlight removal module are optimized using the cycle consistency loss function. The mathematical expression of the cycle consistency loss function is as follows:
[0025]
[0026] In the formula, G represents the highlight generation module, R represents the highlight removal module, E represents the expected value, p represents the data distribution, and I represents the expected value. f This indicates that the original image had no highlights.
[0027] Furthermore, an adversarial loss function is applied to optimize the specular removal module and the specular removal discrimination network. The mathematical expression of the adversarial loss function is as follows:
[0028]
[0029] In the formula, R represents the highlight removal module, and D f This represents the specular removal discriminant network, where E represents the expectation, p represents the data distribution, and I represents the specular removal discriminant network. f This indicates that the original image had no highlights. This indicates a transitional highlight image.
[0030] Furthermore, the input of the reconstruction module includes three RGB channels and a specular mask channel for storing the specular mask information, the mathematical expression of which is as follows:
[0031]
[0032] In the formula, I represents the original image. Indicates a transitional image without highlights, I f This indicates the original image without highlights, where M represents the highlight mask. This indicates a join operation.
[0033] Furthermore, the mathematical expression for the pixel loss function is as follows:
[0034]
[0035] In the formula, E represents the expected value, p represents the data distribution, C represents the reconstruction module, R represents the highlight removal module, G represents the highlight generation module, and I represents the original image. f This indicates that the original image had no highlights.
[0036] Furthermore, the reconstruction module also uses a global loss function to optimize the final specular-free image. The mathematical expression of the global loss function is as follows:
[0037]
[0038] In the formula, C represents the reconstruction module, R represents the highlight removal module, G represents the highlight generation module, E represents the expectation, and p represents the data distribution. This represents a transitional image without highlights, where Ψ is the image dilation function, M represents the highlight mask, I represents the original image, and n represents the number of pixels in the original image.
[0039] Compared with the prior art, the present invention has the following advantages:
[0040] This image specular highlight removal method based on weakly supervised learning completes highlight removal by jointly training highlight generation, highlight removal and reconstruction modules using a recurrent generative adversarial network architecture. During training, the loss function of subsequent modules is fed back to the preceding modules, enabling training to be completed using only the highlight image and achieving good highlight removal results. Compared with traditional algorithms and existing weakly supervised learning methods, it has the advantages of simple operation and good highlight removal effect. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.
[0043] Figure 1 This is a flowchart of the image specular highlight removal method based on weakly supervised learning disclosed in the embodiments;
[0044] Figure 2 It is the result of a qualitative experiment. Detailed Implementation
[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0046] Example:
[0047] like Figure 1As shown, the image specular highlight removal method based on weakly supervised learning includes the following steps:
[0048] First, the highlight region of the image is decomposed using a sparse non-negative matrix, and the image without highlight is cropped from the non-highlight region as training data.
[0049] Next, the training data is input into three joint training modules that are connected end-to-end to perform the tasks of specular generation, specular removal, and image reconstruction, respectively. The network is then trained using a recurrent generative adversarial network architecture to generate the final specular-free image.
[0050] Specifically, the highlight regions of the image are decomposed using a sparse nonnegative matrix, and images without highlights are cropped from the non-highlight regions as training data, including the following steps:
[0051] First, create the following collection:
[0052]
[0053] Next, since the highlights only partially exist, therefore k s (x) is non-zero only in the highlight region, satisfying sparsity. By applying the sparse nonnegative matrix method, the internal dimension of the matrix decomposition is set to 2. While maintaining the l2 norm, k is solved by minimizing the l1 norm. s (x), obtain the specular mask corresponding to the original image, use the specular mask to crop out the original specular image, put the specular mask into the specular mask set, and randomly select a mask that does not overlap with the specular mask in the specular mask set to obtain the original non-spectral image.
[0054] In this equation, the left side represents the original image with a total of n pixels, and the matrix size is 3xn. The 3x2 matrix on the left half of the right side represents the fixed parameters of diffuse and specular reflection in the RGB channels, and the right half is a 2xn matrix containing the diffuse and specular reflection coefficients.
[0055] The above operations can obtain a large number of original highlight images and original non-highlight images from the highlight images. Although the two are not paired, they can be used as training data.
[0056] Specifically, training data is input into three jointly trained modules connected end-to-end to perform highlight generation, highlight removal, and image reconstruction tasks, respectively. The network is trained using a recurrent generative adversarial network architecture to generate the final highlight-free image, including the following steps:
[0057] First, the highlight generation module transforms the input original image without highlights into a transitional highlight image, and then uses a highlight discrimination network to determine whether the transitional highlight image is real or fake.
[0058] Specifically, the network structure of the specular generation module includes three downconvolutional layers for downsampling, a main network consisting of nine residual network blocks, and three upconvolutional layers for upsampling and outputting the image. This network does not contain any pooling layers; instead, it uses stride convolution and deconvolution for upsampling and downsampling. Only the input and output layers are 9x9 kernel convolutional layers; the rest are 3x3 kernel convolutional layers with a stride of 2. Furthermore, except for the residual network and output layers, each layer undergoes spatial normalization and ReLU non-linear activation after each convolutional layer.
[0059] Specifically, the specular discrimination network uses the PatchGAN architecture to determine the authenticity of each 4x4 block in the image, and finally the average of all calculation results is taken.
[0060] Specifically, the specular generation module and the specular discrimination network are optimized based on the objective function of PatchGAN, and the mathematical expression of the objective function is as follows:
[0061]
[0062] In the formula, G represents the specular generation module, and D... h This represents a specular discrimination network, where E represents the expectation, p represents the data distribution, and I represents the specular discrimination network. f Indicates an image without highlights, I h Indicates the original highlight image, I h ~p(I h This indicates that the image was selected from the original specular image data distribution;
[0063] Preferably, to make the transitional highlight image generated by the highlight generation module closer to the original highlight image, thereby reducing the noise generated by the highlight generation module, an identity loss function is applied to optimize the transitional highlight image. Specifically, the mathematical expression of the identity loss function is as follows:
[0064]
[0065] In the formula, G represents the specular generation module, E represents the expected value, p represents the data distribution, and I represents the specular value. h Represents the original specular image, ||G(I h ),I h ||1 represents Norm;
[0066] The identity loss function can reflect the pixel differences between the transitional highlight image generated by the highlight generation module and the original highlight image;
[0067] Next, the highlight removal module takes the transition highlight image as input and removes its highlight parts to obtain the corresponding transition no-highlight image. The highlight removal discrimination network is then used to determine whether the transition no-highlight image is real or fake.
[0068] Preferably, to make the transitional no-highlight image generated by the highlight removal module closer to the original no-highlight image, the highlight removal module and the highlight generation module follow Cycle-GAN, and the cycle consistency loss function is applied to optimize the highlight generation module and the highlight removal module. The mathematical expression of the cycle consistency loss function is as follows:
[0069]
[0070] In the formula, G represents the highlight generation module, R represents the highlight removal module, E represents the expected value, p represents the data distribution, and I represents the expected value. f This indicates that the original image had no highlights.
[0071] Preferably, an adversarial loss function is applied to optimize the specular removal module and the specular removal discriminant network. The mathematical expression of the adversarial loss function is as follows:
[0072]
[0073] In the formula, R represents the highlight removal module, and D f This represents the specular removal discriminant network, where E represents the expectation, p represents the data distribution, and I represents the specular removal discriminant network. f This indicates that the original image had no highlights. This indicates a transitional highlight image.
[0074] Finally, the reconstruction module takes the original image, the transitional no-highlight image, and the highlight mask as input, and uses a pixel loss function to obtain the final no-highlight image.
[0075] Specifically, the input of the reconstruction module includes three RGB channels and a specular mask channel for storing specular mask information, the mathematical expression of which is as follows:
[0076]
[0077] In the formula, I represents the original image. Indicates a transitional image without highlights, I f This indicates the original image without highlights, where M represents the highlight mask. This indicates a join operation.
[0078] Specifically, the mathematical expression for the pixel loss function is as follows:
[0079]
[0080] In the formula, E represents the expected value, p represents the data distribution, C represents the reconstruction module, R represents the highlight removal module, G represents the highlight generation module, and I represents the original image. r I represents the combination of the original image, the transitional image without highlights, and the highlight mask in the above formula. fThis indicates that the original image had no highlights.
[0081] Since the information covered by the highlight area is related to the adjacent areas, preferably, the reconstruction module also uses a global loss function to optimize the final image without highlights. The mathematical expression of the global loss function is as follows:
[0082]
[0083] In the formula, C represents the reconstruction module, R represents the highlight removal module, G represents the highlight generation module, E represents the expectation, and p represents the data distribution. This represents a transitional image without highlights, where Ψ is the image dilation function, M represents the highlight mask, I represents the original image, and n represents the number of pixels in the original image. r This represents the combination of the original image, the transitional image without highlights, and the highlight mask in the above formula.
[0084] This image specular highlight removal method based on weakly supervised learning includes five types of loss functions: adversarial loss function, cycle consistency loss function, identity loss function, pixel loss function, and global loss function. The global loss function is a weighted sum of these five loss functions, and its mathematical expression is as follows:
[0085]
[0086] In the formula, ω represents the weight of each loss function.
[0087] Specifically, the network components of the image specular highlight removal method based on weakly supervised learning consist of a generator network and a discriminator network. The network is optimized by solving a minimization problem, and its mathematical expression is as follows:
[0088] .
[0089] To more directly demonstrate the superiority of this weakly supervised learning-based image specular highlight removal method, a comparative experiment was conducted between this weakly supervised learning-based image specular highlight removal method and existing highlight removal methods.
[0090] In the experiment, the image specular highlight removal method based on weakly supervised learning was tested using a PyTorch deep learning framework (version 1.9.0, CUDA version 10.2) and an NVIDIA GTX 2080 Ti graphics card. Specular images from SHIQ and LIME were used as the original images for training and testing. A Gaussian distribution with a mean of 0 and a standard deviation of 0.02 was used to initialize the model. The model was trained for a total of 100 epochs, with the learning rate fixed at 0.0002 for the first 50 epochs and linearly decaying to 0 in the last 50 epochs. Specifically, during the training data generation process, the same number of original non-spectral images were cropped as the network input, and the three modules in the network (spectral generation module, specular removal module, and image reconstruction module) all adopted a joint training strategy. The overall loss function ω was set to 1, 10, 20, 10, and 10, respectively. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were used as performance metrics for specular removal to evaluate the performance of each method.
[0091] Qualitative experimental results are as shown in the attached instruction manual. Figure 2 As shown. Tan, Akashi, and Yamamoto represent methods based on single images; Yi represents a weakly supervised learning method based on unpaired data; and Shi and Fu represent fully supervised learning methods requiring paired data without specular highlights, specular masks, and specular reflections. (See attached manual.) Figure 2 As can be seen, traditional specular highlight removal methods (such as Tan, Akashi, and Yamamoto) perform poorly when processing natural images, resulting in severe color distortion, especially when white areas are present. Deep learning-based specular highlight removal methods, such as Shi and Fu, perform well in image processing, but they require a large amount of additional information for training, making them complex. In contrast, the image specular highlight removal method proposed in this invention, based on weakly supervised learning, achieves good highlight removal results with only the specular image, making it more widely applicable.
[0092] To avoid the interference of subjective human observation in the judgment of results, we also used two indicators, PSNR and SSIM, to evaluate the performance of each method. The quantitative experimental results are shown in Table 1:
[0093]
[0094] Table 1. Quantitative experimental results of each method
[0095] Different training methods use different data types, where N / A indicates that no training data is needed. Since the proposed image specular highlight removal method based on weakly supervised learning only uses highlight images, it also uses N / A.
[0096] Table 1 shows that the image specular highlight removal method based on weakly supervised learning proposed in this invention achieves the highest SSIM score and the second-best PSNR score, significantly outperforming existing weakly supervised learning methods. While slightly lower than Fu's fully supervised learning method in PSNR, Fu's method requires a large amount of additional data, such as the corresponding non-highlight and specular reflection images of the highlight images, and the PSNR metric cannot effectively reflect the visual quality of the image. The image specular highlight removal method based on weakly supervised learning proposed in this invention also performs well even in the absence of non-highlight images, reducing the dependence on the dataset at a slight performance cost, and thus has wider application value.
[0097] In summary, the image specular highlight removal method based on weakly supervised learning proposed in this invention completes highlight removal by jointly training highlight generation, highlight removal and reconstruction modules in a recurrent generative adversarial network architecture. During the training process, the loss function of subsequent modules is fed back to the preceding modules, enabling training to be completed using only the highlight image and achieving good highlight removal results. Compared with traditional algorithms and existing weakly supervised learning methods, it has the advantages of simple operation and good highlight removal effect.
[0098] Note: Please refer to the following analysis for the multiple existing methods mentioned in the article:
[0099] Shi: In 2017, Shi et al. applied a network model based on an encoder-decoder structure to solve the highlight removal problem. During the training process, the encoder learns the highlight features of the image, and then the decoder removes them. The completed highlight removal model has excellent processing speed and can be applied to real images from the Internet, and even video images.
[0100] Fu et al. proposed a multi-task network for jointly performing specular detection and removal tasks. To eliminate the discrepancy between synthetic training samples and real test images, Fu created a dataset called SHIQ containing tens of thousands of real images.
[0101] Tan et al. from the University of Tokyo first proposed a non-color segmentation specular removal method, which is based solely on the chromaticity information of the image and does not require geometric information. The basic idea is to iteratively compare the logarithmic derivatives of the intensity between the input image and the diffuse image. Their method first normalizes the color information of the image, then solves the mapping relationship between chromaticity and illumination intensity. The reflection coefficient and reflection component can be solved using the maximum diffuse chromaticity value. This method processes a single image entirely, and all operations are performed locally, involving calculations between at most two adjacent pixels. Therefore, it is also suitable for objects with complex surface textures. However, due to the large amount of computation and complexity, the processing time is relatively long.
[0102] Akashi et al. (2014) utilized sparse non-negative matrix factorization (NMF) to solve a two-color reflectance model. This method optimizes the solution of the non-negative matrix, simultaneously estimating surface color and separating reflectance components. Unlike methods that rely on prior conditions to improve reflectance component separation, it does not require prior information or adjustments to numerous hyperparameters, and can handle cases where diffuse reflection color is close to the illumination color. Therefore, it has a wider range of applications, more accurate removal results, and stronger robustness.
[0103] Yamamoto et al. proposed an image enhancement method in 2019 to improve the accuracy of existing specular highlight removal methods. This method first uses an existing specular highlight removal method to obtain the diffuse and specular reflection components in the image, and then applies a highly emphasized filter to both components. Generally, incorrectly separated pixels will have larger response values after passing through the highly emphasized filter. This method can be used to detect incorrectly separated images. Then, leveraging the similarity between the target pixel and the reference pixel, the incorrectly separated result is replaced with another reference pixel.
[0104] Yi et al. proposed a deep neural network model to extract specular reflection components from faces, and then backtrack these components in the original scene to obtain environmental information. Since real training data for specular highlight extraction is very limited, this method first pre-trains the network using synthetic images, and then fine-tunes the network using an unsupervised strategy to address the performance degradation caused by the mismatch between real and synthetic data. Compared to supervised learning only on synthetic images, the unsupervised learning strategy significantly improves specular highlight removal. Subsequently, Yi et al. further extended this work, proposing an image representation method based on local color distribution, which reduces the network's sensitivity to local misalignment in multi-view images during training. Then, an unsupervised learning method is used to separate the reflection components using a multi-view dataset.
[0105] It should be understood that the terms "first," "second," etc., are used in this invention to describe various information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this invention, "first" information can also be referred to as "second" information, and similarly, "second" information can also be referred to as "first" information. In addition, the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.
[0106] The above description provides one or more embodiments in conjunction with specific content, and does not imply that the specific implementation of the present invention is limited to these descriptions. Any methods or structures that are similar to or identical to those of the present invention, or any technical deductions or substitutions made based on the concept of the present invention, should be considered as protected by the present invention.
Claims
1. A method for removing specular highlights from images based on weakly supervised learning, characterized in that, Includes the following steps: First, create the following collection: Next, the internal dimension of the matrix decomposition is set to 2, and k is solved by minimizing the l1 norm while maintaining the l2 norm. s (x), obtain the specular mask corresponding to the original image, use the specular mask to crop out the original specular image, and put the specular mask into the specular mask set. Randomly select a mask that does not overlap with the specular mask in the specular mask set to obtain the original non-spectral image. In the formula, the left side of the equation represents the original image with a total number of pixels of n, and the matrix size is 3xn. The 3x2 matrix in the left half of the right side of the equation represents the fixed parameters of diffuse reflection and specular reflection on the RGB channels, and the right half is a 2xn matrix containing the diffuse reflection and specular reflection coefficients. Next, the training data is input into three joint training modules that are connected end-to-end to perform highlight generation, highlight removal and image reconstruction tasks respectively, and the network is trained using a recurrent generative adversarial network architecture to generate the final highlight-free image; The training data is input into three jointly trained modules connected end-to-end to perform specular generation, specular removal, and image reconstruction tasks, respectively. The network is trained using a recurrent generative adversarial network architecture to generate the final specular-free image, including the following steps: First, the highlight generation module converts the input original image without highlights into a transitional highlight image, and uses a highlight discrimination network to determine the authenticity of the transitional highlight image; Next, the highlight removal module inputs the transition highlight image and removes its highlight portion to obtain the corresponding transition no-highlight image. The highlight removal discrimination network is then used to determine the authenticity of the transition no-highlight image. Finally, the reconstruction module takes the original image, the transitional no-highlight image, and the highlight mask as input, and uses a pixel loss function to obtain the final no-highlight image. The input of the reconstruction module includes three RGB channels and a specular mask channel for storing the specular mask information, the mathematical expression of which is as follows: In the formula, I represents the original image. This indicates an image with no highlights during transition. This indicates the original image without specular highlights, M represents the specular mask, and ⊕ represents the connection operation.
2. The image specular highlight removal method based on weakly supervised learning according to claim 1, characterized in that, The transition highlight image is optimized using the identity loss function, the mathematical expression of which is as follows: In the formula, G This indicates the specular highlight generation module. E Indicates taking the expected value. p Indicates data distribution, I h Represents the original specular image, ||G(I h ),I h ||1 represents the l1 norm.
3. The image specular highlight removal method based on weakly supervised learning according to claim 1, characterized in that, The specular discrimination network uses the PatchGAN architecture to determine the authenticity of each 4x4 block in the transition specular image, and finally the average of all calculation results is taken. The specular generation module and the specular discrimination network are optimized based on the objective function of PatchGAN, and the mathematical expression of the objective function is as follows: In the formula, G This indicates the specular highlight generation module. D h This indicates a specular discrimination network. E Indicates taking the expected value. p Indicates data distribution, This indicates that the original image had no highlights. I h This represents the original highlight image. I h ~p(I h ) This indicates that the image was selected from the original specular image data distribution.
4. The image specular highlight removal method based on weakly supervised learning according to claim 1, characterized in that, The highlight generation module and the highlight removal module are optimized using the cycle consistency loss function. The mathematical expression of the cycle consistency loss function is as follows: In the formula, G R represents the highlight generation module, and R represents the highlight removal module. E Indicates taking the expected value. p Indicates data distribution, This indicates that the original image had no highlights.
5. The image specular highlight removal method based on weakly supervised learning according to claim 1, characterized in that, The adversarial loss function is applied to optimize the specular removal module and the specular removal discrimination network. The mathematical expression of the adversarial loss function is as follows: In the formula, R represents the highlight removal module, and D... f This indicates the specular removal discrimination network. E Indicates taking the expected value. p Indicates data distribution, This indicates that the original image had no highlights. This indicates a transitional highlight image.
6. The image specular highlight removal method based on weakly supervised learning according to claim 1, characterized in that, The mathematical expression for the pixel loss function is as follows: In the formula, E Indicates taking the expected value. p Indicates the data distribution, C represents the reconstruction module, and R represents the highlight removal module. G This represents the highlight generation module, and I represents the original image. This indicates that the original image had no highlights.
7. The image specular highlight removal method based on weakly supervised learning according to claim 1, characterized in that, The reconstruction module also uses a global loss function to optimize the final image without highlights. The mathematical expression of the global loss function is as follows: In the formula, C represents the reconstruction module, and R represents the highlight removal module. G This indicates the specular highlight generation module. E Indicates taking the expected value. p Indicates data distribution, This represents a transitional image without highlights, where Ψ is the image dilation function, M represents the highlight mask, I represents the original image, and n represents the number of pixels in the original image.