Metal part surface highlight removal method, model construction method, and computer readable medium

Abstract

The application discloses a metal part surface highlight removal model construction method, comprising the following steps: step one: constructing a highlight detection network; 11) inputting a metal part highlight image I0 into the highlight detection network and obtaining a highlight mask image M out ; 12) calculating a detection loss function, and judging whether the detection loss function is less than a set threshold value; if yes, step 21) is executed; if no, the weight is updated, and step 11) is executed; step two: constructing a highlight removal network; 21) inputting a fusion image into the highlight removal network, simultaneously extracting the features of the fusion image in a spatial domain and a frequency domain through fast Fourier convolution, and generating a highlight-free image I out ; 22) calculating a loss function, and judging whether the loss function is less than a set threshold value; if yes, a metal part surface highlight removal model is obtained; if no, the weight is updated, and step 21) is executed. The application further discloses a metal part surface highlight removal method and a computer readable medium.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a method for removing highlights from the surface of metal parts, a model construction method, and a computer-readable medium. Background Technology

[0002] Spectra are widespread in natural images. Spectra on object surfaces disrupt image continuity, creating false edges; they also reduce local contrast, leading to the loss of texture and color information. Spectra removal techniques can improve image quality and are beneficial for visual tasks such as image segmentation, stereo matching, text recognition, and object detection. Due to the Fresnel properties of their material, metal parts in industrial environments are prone to producing large areas of high-intensity spectra, severely hindering machine vision-based applications such as geometric feature detection, dimensional measurement, and 3D reconstruction of metal parts. Therefore, spectra removal from metal part surfaces is essential.

[0003] Currently, image highlight removal methods include hardware-based and image processing-based methods. Hardware-based methods suppress highlights by introducing polarization devices during image acquisition. This method increases the complexity of practical operation, requiring continuous adjustment of the polarizing lens angle to remove polarized light reflected from the object surface to achieve highlight removal. However, due to the high reflectivity of metal parts, simply adjusting the polarization angle cannot completely eliminate the highlight areas. Furthermore, the introduction of polarization devices cannot adapt to varying lighting conditions in industrial environments, making it difficult to ensure that both dark and bright areas of the metal part surface are simultaneously captured with high precision, resulting in poor practical application performance. Therefore, this type of method is not suitable for highlight removal on metal parts.

[0004] Image processing-based methods include two-color reflectance models, image inpainting, and deep learning. Two-color reflectance models treat surface reflection as a combination of diffuse and specular reflection. By removing specular reflections and preserving diffuse reflections, specular highlights are removed, thus achieving highlight removal. This type of method assumes rich chromaticity information on the object's surface and the presence of diffuse pixels in the highlight areas. However, metal parts typically have monochromatic surfaces lacking necessary chromaticity information; furthermore, their high reflectivity and simple texture mean that highlights are almost entirely generated by specular reflection, with no diffuse reflection. Therefore, metal parts do not meet the model's assumptions, making two-color reflectance models unsuitable for highlight removal. Image inpainting highlight removal methods first convert between different color spaces such as RGB, HSV, and HIS, then use clustering algorithms to locate highlight areas in the image. Next, using the color and intensity of neighboring pixels as a reference, spatial pixel interpolation is used to correct the highlight pixels, achieving highlight removal. This type of method suffers from difficulties in setting clustering thresholds, making it impossible to effectively distinguish between highlight and white areas on an object's surface. It inevitably clusters some white areas as highlight areas. For example, when detecting highlights on the surface of a silver-white aluminum metal part, the interference of surface color may prevent accurate location of the highlight area. Furthermore, due to limitations in spatial neighborhood interpolation, this method is typically only suitable for removing highlights from small areas of less than 10% of the image. However, the highlight areas on metal parts are large and intense. Image-based highlight removal methods can lead to color distortion and texture loss in the restored highlight areas, hindering machine vision-based geometric feature detection of metal parts. For instance, color-distorted restored areas often exhibit false edges during edge detection, reducing the accuracy of geometric feature detection. Deep learning-based image highlight removal methods using convolutional neural networks differ from the above two methods. They do not require building a reflection model of the object's surface. Instead, they use a data-driven approach, extracting features from the highlight image through convolutional operations and iteratively learning the differences between the highlight and non-highlight images to achieve the conversion from a highlight image to a non-highlight image. Current image highlight removal models effectively remove highlights from ordinary images, such as those on common everyday items like wooden furniture, fruit, and plush toys. However, compared to these everyday items, metal parts, due to their high surface reflectivity, are prone to producing high-intensity, large-area highlights. This results in a greater frequency domain difference between highlight-free and highlight-free images of metal parts. For example, highlight-free images have fewer high-frequency features and more low-frequency features. However, existing image highlight removal models, such as neural networks based on traditional convolution or dilated convolution, extract features in the spatial domain of the image and do not learn the difference between highlight and highlight-free images from the frequency domain perspective. Therefore, they cannot be effectively applied to highlight removal from the surface of metal parts. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method for removing highlights from the surface of metal parts, a model construction method, and a computer-readable medium. The method extracts the spatial and frequency domain features of the highlight image of the metal part, learns the spatial and frequency domain differences between the highlight image and the non-highlight image, realizes the conversion between the two, and completes the highlight removal from the surface of the metal part.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] This invention first proposes a method for constructing a speckle removal model for metal parts, comprising the following steps:

[0008] Step 1: Constructing a High-Light Detection Network

[0009] 11) Input the specular image I0 of the metal part into the specular detection network and obtain the specular mask image M. out ;

[0010] 12) Calculate the specular mask image M out Compared with the real specular mask image M gt The detection loss function is used to determine whether the value of the detection loss function has converged: if yes, then step 21) is executed; if no, then the weights of the specular detection network are updated through backpropagation of the error, and step 11) is executed.

[0011] Step 2: Construct a highlight removal network

[0012] 21) Fusing the specular image I0 of the metal part and the specular mask image M out The features of the image are extracted and a fused image is obtained. The fused image is then input into a specular removal network, which simultaneously extracts features in the spatial and frequency domains of the fused image through Fast Fourier Convolution, generating a specular-free image I. out ;

[0013] 22) Calculate the image I without highlights out Compared to a real image without highlights, I gt The loss function is used to determine whether the value of the loss function has converged: if yes, the specular removal model of the metal part surface is obtained; if not, the weights of the specular removal network are updated through backpropagation of the error, and step 21 is executed.

[0014] Furthermore, the specular detection network includes an encoder, a feature processing module, and a decoder;

[0015] The encoder uses three conventional convolutions to downsample the input metal part highlight image I0 to reduce image dimensionality and increase the image's abstract semantics.

[0016] The feature processing module uses six traditional convolutional residual blocks to extract image features at depth;

[0017] The decoder uses three upsampled samples corresponding to those in the encoder to decode the extracted features, restoring the original image size and obtaining the specular mask image M. out .

[0018] Furthermore, the detection loss function adopts the L1 loss function:

[0019] Loss NetD =|M out -M gt ||1

[0020] Among them, Loss NetD M represents the detection loss function; out Represents a specular mask image; M gt This represents the actual specular mask image.

[0021] Furthermore, the highlight removal network includes a generator and a discriminator;

[0022] The generator is used to extract features of the fused image in the spatial and frequency domains and generate a specular-free image I. out ;

[0023] The discriminator is used to identify whether each small patch in the input image comes from the image without highlights I. out Or it comes from a real image without highlights I gt ;

[0024] By having the generator and discriminator work against each other, the highlight removal network generates high-quality, highlight-free images.

[0025] Furthermore, the generator includes an encoder, a feature processing module, and a decoder;

[0026] The encoder uses three fast Fourier convolutions to downsample the fused image, and after each downsampling, it uses one fast Fourier convolution residual block to encode spatial features and frequency domain features of the fused image.

[0027] The feature processing module uses six Fast Fourier Convolutional residual blocks to deeply extract the encoded spatial and frequency domain features;

[0028] The decoder uses three upsampling operations corresponding to the encoder. After each upsampling, a Fast Fourier Convolutional Residual Block is used to decode the spatial and frequency domain features extracted from the depth. Simultaneously, through skip connections, the low-dimensional features from the encoder are fused in the decoder, and a specular-free image I is output. out .

[0029] Furthermore, the discriminator uses a patch-based discrimination network from the pix2pix model to identify whether each small patch in the input image comes from the specular-free image I generated by the generator. out Or it comes from a real image without highlights I gt .

[0030] Furthermore, the generator employs a repair loss function, which is the sum of the L1 loss function, the perceptual loss function, and the feature matching loss function:

[0031] Loss L1 =10×(1-M) out )×||I out -I gt ||1

[0032] Loss PL =0.02×||ψ(I) out )-ψ(I gt )||1

[0033] Loss FM =100×MSE(D(I) out ), D(I gt ))

[0034] Among them, Loss L1 Represents the L1 loss function; Loss PL Represents the perceptual loss function; Loss FM I represents the feature matching loss function; out Indicates an image without highlights; I gt Indicates a true image without highlights; M out ψ(·) represents the specular mask image; ψ(·) represents the feature map extracted by the VGG19 network; MSE represents the mean square error; D(·) represents the feature map extracted by the discriminator.

[0035] The discriminator employs an adversarial loss function, which is the sum of the GAN loss function with local adversarial loss and the gradient penalty loss function.

[0036] Loss partGAN =10×[-E[M out ⊙D(I out )]-E[(1-M out )⊙D(1-I out )]]+10×[-E[D(I out )]-E[D(I gt )]]

[0037]

[0038] Among them, Loss partGAN GAN loss function representing local adversarial loss; Loss GP I represents the gradient penalty loss function; out Indicates an image without highlights; I gt Indicates a true image without highlights; M out denoted as the specular mask image; E represents the expectation; D(·) represents the feature map extracted by the discriminator; The symbol '-' indicates calculating the gradient; '⊙' indicates multiplying the pixel values ​​at the corresponding positions.

[0039] Furthermore, image I without highlights out Compared to a real image without highlights, I gt The loss function between them is the sum of the repair loss function and the adversarial loss function:

[0040] Loss NetR =Loss L1 +Loss PL +Loss FM +Loss partGAN +Loss GP

[0041] Among them, Loss NetR Image I without highlights out Compared to a real image without highlights, I gt The loss function between them.

[0042] This invention also proposes a method for removing highlights from the surface of metal parts. The method involves inputting a highlight image I0 of the metal part into a highlight removal model constructed using the method described above, and outputting a highlight-free image I. out .

[0043] The present invention also proposes a computer-readable medium storing a computer program, which, when executed by a processor, implements the steps of the method for removing highlights from the surface of metal parts as described above.

[0044] The beneficial effects of this invention are as follows:

[0045] Due to varying lighting conditions in industrial environments, metal parts surfaces can develop highlights of varying sizes. To adapt to these lighting conditions, the metal part surface highlight removal model construction method of this invention employs a highlight detection network to locate highlight regions of different sizes and generate highlight mask images. This improves the accuracy of highlight removal from metal part surfaces and has greater application value. In machine vision-based metal part geometric feature detection, it can be used as image preprocessing to improve image quality and ensure subsequent detection accuracy. For example, processing the highlight image of a metal part can eliminate false edges caused by highlights. Simultaneously, a highlight removal network is used, leveraging the global receptive field advantage of Fast Fourier Convolution. Through point-by-point updates in the frequency domain, it generates a global impact on image features. It can also extract the spatial and frequency domain features of the highlight image of the metal part, learn the differences between the highlight image and the non-highlight image in the spatial and frequency domains, realize the conversion from the highlight image to the non-highlight image, and complete the highlight removal from the metal part surface. Attached Figure Description

[0046] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration:

[0047] Figure 1 This is a schematic diagram illustrating the principle of the method for constructing a model for removing highlights from the surface of metal parts according to the present invention.

[0048] Figure 2 A schematic diagram illustrating the principle of extracting spatial and frequency domain features using Fast Fourier Convolution;

[0049] Figure 3 Image comparison of the process of speckle removal from high-reflectivity metal parts;

[0050] Figure 4 Image comparison of removing highlights from metal parts using different modeling methods;

[0051] Figure 5 A comparison curve of PSNR metric parameters for different method models;

[0052] Figure 6 A comparison graph of SSIM metric parameters for different method models;

[0053] Figure 7 This image comparison shows the results of Sobel edge detection on the surface repair images of metal parts under different method models. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0055] like Figure 1 As shown, the method for constructing a speckle removal model for metal parts in this embodiment includes the following steps:

[0056] Step 1: Constructing a High-Light Detection Network

[0057] A specular detection network locates highlights in an image of a metal part and generates a specular mask image, with the metal part's specular image as input. This embodiment's specular detection network includes an encoder, a feature processing module, and a decoder. The encoder uses three conventional convolutions to downsample the input metal part specular image I0 to reduce image dimensionality and increase the image's abstract semantics. The feature processing module uses six conventional convolutional residual blocks to extract image features at depth. The decoder uses three upsampling operations corresponding to the encoder to decode the extracted features, restoring the original image size and obtaining the specular mask image M. out .

[0058] 11) Input the specular image I0 of the metal part into the specular detection network and obtain the specular mask image M. out ;

[0059] 12) Calculate the specular mask image M out Compared with the real specular mask image M gt The detection loss function is used to determine whether the value of the detection loss function has converged: if yes, then step 21 is executed; if no, then the weights of the specular detection network are updated through backpropagation of the error, and step 11 is executed.

[0060] In this embodiment, the detection loss function adopts the L1 loss function:

[0061] Zoss NetD =||M out -M gt ||1

[0062] Among them, Loss NetD M represents the detection loss function; out Represents a specular mask image; M gt This represents the actual specular mask image.

[0063] Step 2: Construct a highlight removal network

[0064] The specular removal network removes highlights from a metal part image and generates a specular-free image. The input is a depth-fused image of the specular image and the specular mask image. This embodiment's specular removal network includes a generator and a discriminator. Specifically, the generator extracts features from the fused image in the spatial and frequency domains and generates the specular-free image I. outThe generator in this embodiment includes an encoder, a feature processing module, and a decoder. The encoder uses three Fast Fourier Convolutions (FFTs) to downsample the fused image. After each downsampling, one FFT residual block is used to encode spatial and frequency domain features of the fused image. The feature processing module uses six FFT residual blocks to extract the encoded spatial and frequency domain features. The decoder uses three upsamplings corresponding to the encoder. After each upsampling, one FFT residual block is used to decode the extracted spatial and frequency domain features. Simultaneously, through skip connections, the low-dimensional features from the encoder are fused in the decoder, and a specular-free image I is output. out Fast Fourier convolution performs a fast Fourier transform on each channel, providing a global receptive field that covers the entire image. For example... Figure 2 As shown, Fourier convolution divides the channel into two parallel branches: one branch uses traditional convolution to extract features in the spatial domain; the other branch uses fast Fourier transform to extract features in the frequency domain, and finally the features are fused for output.

[0065] The discriminator in this embodiment is used to identify whether each small patch in the input image comes from the image without highlights. out Or it comes from a real image without highlights I gt Through the interaction between the generator and the discriminator, the specular removal network generates high-quality specular-free images. In this embodiment, the discriminator uses a patch-based discriminator network from the pix2pix model to determine whether each small patch in the input image originates from the specular-free image I generated by the generator. out Or it comes from a real image without highlights I gt .

[0066] 21) Fusing the specular image I0 of the metal part and the specular mask image M out The features of the image are extracted and a fused image is obtained. The fused image is then input into a specular removal network, which simultaneously extracts features in the spatial and frequency domains of the fused image through Fast Fourier Convolution, generating a specular-free image I. out ;

[0067] 22) Calculate the image I without highlights out Compared to a real image without highlights, I gt The loss function is used to determine whether the value of the loss function has converged: if yes, the specular removal model of the metal part surface is obtained; if not, the weights of the specular removal network are updated through backpropagation of the error, and step 21 is executed.

[0068] In this embodiment, the generator employs a repair loss function, which is the sum of the L1 loss function, the perceptual loss function, and the feature matching loss function:

[0069] LossL1 =10×(1-M) out )×||I out -I gt ||1

[0070] Loss PL =0.02×||ψ(I) out )-ψ(I gt )||1

[0071] Loss FM =100×MSE(D(I) out ), D(I gt ))

[0072] Among them, Loss L1 Represents the L1 loss function; Loss PL Represents the perceptual loss function; Loss FM I represents the feature matching loss function; out Indicates an image without highlights; I gt Indicates a true image without highlights; M out ψ(·) represents the specular mask image; ψ(·) represents the feature map extracted by the VGG19 network pre-trained on ImageNet; MSE represents the mean squared error; D(·) represents the feature map extracted by the discriminator.

[0073] In this embodiment, the discriminator employs an adversarial loss function, which is the sum of the GAN loss function with local adversarial loss and the gradient penalty loss function:

[0074] Loss partGAN =10×[-E[M out ⊙D(I out )]-E[(1-M out )⊙D(1-I out )]]+10×[-E[D(I out )]-E[D(I gt )]]

[0075]

[0076] Among them, Loss partGAN GAN loss function representing local adversarial loss; Loss GP I represents the gradient penalty loss function; out Indicates an image without highlights; I gt Indicates a true image without highlights; M out denoted as the specular mask image; E represents the expectation; D(·) represents the feature map extracted by the discriminator; The symbol '-' indicates calculating the gradient; '⊙' indicates multiplying the pixel values ​​at the corresponding positions.

[0077] The GAN loss function with local adversarial loss aims to recover the texture information of the metallic highlight region as much as possible; the gradient penalty loss function aims to stabilize the training of the discriminator.

[0078] In this embodiment, there is no highlight image I. out Compared to a real image without highlights, I gt The loss function between them is the sum of the repair loss function and the adversarial loss function:

[0079] Loss NetR =Loss L1 +Loss PL +Loss FM +Loss partGAN +Loss GP

[0080] Among them, Loss NetR Image I without highlights out Compared to a real image without highlights, I gt The loss function between them.

[0081] This embodiment also proposes a method for removing highlights from the surface of metal parts. The method involves inputting the highlight image I0 of the metal part into the highlight removal model constructed using the method described above, and outputting a highlight-free image I. out .

[0082] This embodiment also proposes a computer-readable medium storing a computer program, which, when executed by a processor, implements the steps of the method for removing the gloss from the surface of metal parts as described above.

[0083] Experimental verification

[0084] The following specific experiments will verify the method for removing the gloss from the surface of metal parts in this embodiment.

[0085] The highlight removal model for metal parts based on Fast Fourier Transform (FFT) convolution was trained on the public dataset SHIQ, which contains approximately 40,000 image pairs, each 200×200 pixels. The highlight removal steps for metal images are as follows:

[0086] (1) Input the highlight image of the metal part into the highlight detection network, and generate the corresponding highlight mask image through the encoder, feature extraction module and decoder;

[0087] (2) The highlight image and the highlight mask image generated in step 1 are fused in depth and input into the highlight removal network to generate a highlight-free image;

[0088] (3) Calculate the corresponding loss value based on the results of the above steps, perform error backpropagation, and use the Adam optimizer to update the network weights. The Adam optimizer learning law of the highlight detection network is set to 0.001; the Adam optimizer learning law of the generator in the highlight removal network is set to 0.001, and the Adam optimizer learning law of the discriminator is set to 0.0002.

[0089] The processing procedure of the specular detection network is as follows:

[0090] ① The image with highlights is scaled down to 256×256 and used as input to the highlight detection network. After three downsampling operations, feature encoding is performed to increase abstract semantics.

[0091] ② Use 6 traditional convolutional residual blocks to extract and fuse the encoded features;

[0092] ③ The fused feature image is upsampled three times and decoded to restore the original image size, generating a specular mask image.

[0093] The process of the highlight removal network is as follows:

[0094] ① After fusing the highlight image and the highlight mask image in depth, a fused image is obtained. The fused image is used as input, and after three downsampling steps, the image is encoded. The spatial and frequency domain features of the image are encoded by fast Fourier convolution.

[0095] ②The spatial and frequency domain features of the encoded data are extracted through the depth of six Fast Fourier Convolution residual blocks;

[0096] ③ By using an upsampling layer symmetrical to the downsampling layer, spatial and frequency domain features are decoded to restore the image size. During the decoding process, feature maps from corresponding layers in the downsampling layer are fused through skip connections.

[0097] ④ Finally, the image depth is adjusted through a convolution layer to obtain an image without highlights.

[0098] The loss in the specular removal network is calculated from the restoration loss and the adversarial loss, and the specific calculation formula is as follows:

[0099] Loss G =Loss L1 +Loss PL +Loss FM +10×E[-D(I out )]

[0100]

[0101] like Figure 3As shown, sheet metal part a and metal container b were used as experimental subjects. High-intensity, large-area highlights were generated on the surface of sheet metal part a, while highlight areas of different sizes were generated on the surface of metal container b. The first column is the highlight image of the metal part, the second column is the generated highlight mask image, the third column is the real mask image, the fourth column is the generated image without highlights, and the last column is the real image without highlights.

[0102] The effectiveness of this embodiment is verified by comparing three image highlight removal methods, including the two-color model highlight removal method proposed by Tan in 2007 and Shen in 2014, and the deep learning method using CycleGAN proposed by Funke in 2018. The comparison results are as follows: Figure 4 As shown, Tan's algorithm produced severe color distortion on the surface of metal parts; Shen's algorithm failed to remove highlights from the surface of metal parts; Funke's algorithm produced highlight residue in the high-intensity, large-area highlight repair of sheet metal parts, and obvious local distortion in the highlight repair of different areas of metal containers. This embodiment, however, achieved good visual results on both sheet metal parts and metal containers.

[0103] Quantitative analysis of the embodiments was performed using two metrics: PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index). PSNR is commonly used to measure the satisfaction of an image processing procedure. SSIM measures the similarity between two images, evaluating similarity in terms of brightness, contrast, and structure. The analysis results are shown in Table 1. The present invention achieved the highest scores on both objective metrics. Figure 5 and Figure 6 The comparison results of PSNR and SSIM for sheet metal part a and metal container b under different algorithms allow for a more intuitive observation of the superior performance of the present invention.

[0104] Table 1. Evaluation metrics for the examples (PSNR dB / SSIM%)

[0105] Example Tan2007 Shen2014 Funke2018 This invention Sheet metal parts (a) 10.530 / 47.6 11.717 / 73.1 13.653 / 79.1 24.595 / 91.8 Metal container (b) 11.655 / 43.9 18.904 / 88.6 27.277 / 91.4 32.610 / 94.6

[0106] Compared to the traditional image processing algorithm Tan2007, the image restoration accuracy in this embodiment is improved by 14.065dB to 20.953dB in PSNR and by 44.2% to 50.7% in SSIM; compared to Shen2014, the image restoration accuracy is improved by 12.878dB to 13.706dB in PSNR and by 6.0% to 18.7% in SSIM; compared to the deep learning highlight removal algorithm Funke2018, the image restoration accuracy is improved by 5.333dB to 10.942dB in PSNR and by 3.2% to 12.7% in SSIM.

[0107] To better illustrate the benefits of this invention for geometric feature detection of metal parts based on machine vision, edge detection was performed on images with highlight removal using different algorithms. The edge detection algorithm used the Sobel operator, and the results are as follows. Figure 7 The image shows the Sobel edge detection results for metal part surface restoration images under different algorithms. The first column is the original highlight image, the second column is the detection result without highlight removal, the third column is the edge result after highlight removal using Funke2018, and the fourth column is the detection result after highlight removal using the present invention. Since the Tan2007 and Shen2014 algorithms are significantly less effective at highlight removal on metal parts, only the Funke2018 algorithm was selected for comparison. Figure 7 As can be seen from the selected local area, the highlights disrupt the continuity of the metal part's surface, creating false edges that severely affect the detection of the part's geometric features. Although the Funke2018 algorithm can eliminate some of the false edges, the elimination effect is not good, and a small number of false edge pixels remain. In contrast, this invention can completely eliminate the false edge pixels caused by highlights, improving image quality and ensuring the accuracy of subsequent geometric feature detection of the part.

[0108] from Figure 3 , Figure 4 The image results demonstrate that this invention can effectively remove highlights from metallic images. While removing highlights, it can preserve details in the original highlight image relatively well, and no significant image distortion occurs in the repaired highlight areas. As shown in Table 1, this invention achieved the highest scores on both objective evaluation metrics, PSNR and SSIM, demonstrating superior highlight removal performance. Figure 7 The image results show that the present invention can effectively eliminate false edges caused by highlights, improve image quality, and ensure the accuracy of subsequent machine vision-based geometric feature detection of metal parts. Therefore, the present invention can effectively remove highlights from the surface of metal parts, and the generated metal highlight-free images are close to real highlight-free images both qualitatively and quantitatively, which is beneficial to machine vision-based geometric feature detection tasks of metal parts.

[0109] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A method for constructing a model for removing highlights from the surface of metal parts, characterized in that: Includes the following steps: Step 1: Constructing a High-Light Detection Network 11) Highlighting the metal parts in the image Input the specular detection network and obtain the specular mask image. ; 12) Calculate the specular mask image Compared with real specular mask images The detection loss function is used to determine whether the value of the detection loss function has converged: if yes, then proceed to step 21); if no, then update the weights of the specular detection network through backpropagation of the error and proceed to step 11). Step 2: Construct a highlight removal network 21) Fusing highlight images of metal parts and specular mask image The features of the image are analyzed to obtain a fused image. This fused image is then input into a specular removal network, which simultaneously extracts features in both the spatial and frequency domains using Fast Fourier Convolution. The frequency domain feature extraction is optimized to address the low-frequency differences between the specular and non-spectral images of the metal parts, generating a non-spectral image. Fast Fourier Transform (FFT) performs Fast Fourier Transform on each channel, providing a global receptive field that covers the entire image. FFT divides each channel into two parallel branches: one branch uses conventional convolution for spatial domain feature extraction, and the other branch uses FFT for frequency domain feature extraction. Finally, the features are fused for output. 22) Calculate images without highlights Compared to real images without highlights The loss function is used to determine whether the value of the loss function has converged: if it has, the specular removal model of the metal part surface is obtained; if not, the weights of the specular removal network are updated through backpropagation of the error, and step 21 is executed. The highlight removal network includes a generator and a discriminator; The generator is used to extract features of the fused image in the spatial and frequency domains and generate a specular-free image. ; The discriminator is used to identify whether each small patch in the input image comes from an image without highlights. Or it comes from a real image without highlights. ; By having the generator and discriminator work against each other, the specular removal network generates high-quality specular-free images. The generator employs a repair loss function, which is: The sum of the loss function, the perceptual loss function, and the feature matching loss function: in, express Loss function; Represents the perceptual loss function; This represents the feature matching loss function; Indicates an image without highlights; This represents a true image without highlights; Represents a specular mask image; Indicates passage Feature maps extracted by the network; Indicates mean square error; This represents the feature map extracted by the discriminator; The discriminator employs an adversarial loss function, which is a local adversarial loss. The sum of the loss function and the gradient penalty loss function: in, Indicates localized combat losses Loss function; This represents the gradient penalty loss function; Indicates an image without highlights; This represents a true image without highlights; Represents a specular mask image; This indicates a demand for expectation; This represents the feature map extracted by the discriminator; This indicates calculating the gradient; This indicates that the pixel values ​​at corresponding positions are multiplied. Image without highlights Compared to real images without highlights The loss function between them is the sum of the repair loss function and the adversarial loss function: in, Indicates an image without highlights Compared to real images without highlights The loss function between them.

2. The method for constructing a model for removing highlights from the surface of metal parts according to claim 1, characterized in that: The specular detection network includes an encoder, a feature processing module, and a decoder; The encoder uses three conventional convolutions on the input highlight image of the metal part. Image downsampling is performed to reduce image dimensionality while increasing the image's abstract semantics; The feature processing module uses six traditional convolutional residual blocks to extract image features at depth; The decoder uses three upsampled samples corresponding to those in the encoder to decode the extracted features, restore the original image size, and obtain the specular mask image. .

3. The method for constructing a model for removing highlights from the surface of metal parts according to claim 1 or 2, characterized in that: The detection loss function adopts Loss function: in, Represents the detection loss function; Represents a specular mask image; This represents the actual specular mask image.

4. The method for constructing a model for removing highlights from the surface of metal parts according to claim 1, characterized in that: The generator includes an encoder, a feature processing module, and a decoder; The encoder uses three fast Fourier convolutions to downsample the fused image, and after each downsampling, it uses one fast Fourier convolution residual block to encode spatial features and frequency domain features of the fused image. The feature processing module uses six Fast Fourier Convolutional residual blocks to deeply extract the encoded spatial and frequency domain features; The decoder uses three upsampling operations corresponding to the encoder. After each upsampling, a Fast Fourier Convolutional Residual Block is used to decode the spatial and frequency domain features extracted from the depth. Simultaneously, through skip connections, the low-dimensional features from the encoder are fused in the decoder, and a specular-free image is output. .

5. The method for constructing a surface highlight removal model for metal parts according to claim 1, characterized in that: The discriminator uses a patch-based discrimination network from the pix2pix model to determine whether each small patch in the input image comes from a specular-free image generated by the generator. Or it comes from a real image without highlights. .

6. A method for removing highlights from the surface of metal parts, characterized in that: Highlight image of metal parts Input the highlight removal model of the metal part surface constructed by the method described in any one of claims 1-5, and output an image without highlights. .

7. A computer-readable medium, characterized in that: The computer-readable medium stores a computer program that, when executed by a processor, implements the steps of the method for removing gloss from the surface of a metal part as described in claim 6.

Images