Method and system for removing reflected light of workpiece surface image based on diffusion model

Abstract

The invention discloses a workpiece surface image reflection removal method and system based on a diffusion model, and the method comprises the steps: carrying out the shooting and data preprocessing of a to-be-processed workpiece through an industrial camera, and obtaining a to-be-processed workpiece surface data image; based on the de-noising diffusion probability model, introducing a conditional guidance branch structure, and constructing a workpiece image reflection removal network; and inputting a to-be-processed workpiece surface data image into the workpiece image reflection removal network for reflection removal to obtain a workpiece surface image reflection removal result. According to the embodiment of the invention, the condition mixed distribution can be learned in the reverse denoising process to generate the workpiece surface non-reflective image, so that the precision of removing the reflective light of the workpiece surface image is improved. The method can be widely applied to the technical field of computer visual identification.

Description

Technical Field

[0001] This application relates to the field of computer vision recognition technology, and in particular to a method and system for removing reflections from workpiece surface images based on a diffusion model. Background Art

[0002] Reflections on workpiece surfaces manifest in two main forms: highlighted areas and specular reflections. For the first type, highlighted areas, current methods use visual lighting equipment to remove them. These devices are primarily categorized into low-angle and polarized light sources, with low-angle sources limited by the specific scene's light source height. Polarized light sources, on the other hand, can cause significant differences in polarization depending on the workpiece surface material. While this method can remove highlights, the polarizing filter must be matched to the camera lens; changing the lens necessitates changing the polarizing filter, and the polarizing filter at the light source may slip, impacting production efficiency in practice. Furthermore, since specular reflections cannot be removed by changing the lighting method, current methods for removing highlighted areas cannot solve the problem of specular reflections on workpiece surfaces.

[0003] Currently, AI-based image specular reflection removal methods primarily employ image generation strategies to generate non-reflective images of workpiece surfaces. However, the biggest problem with using Generative Adversarial Networks (GANs) to generate non-reflective images is that the discriminator in the GAN is prone to mode collapse during model training. This means the generator becomes trapped in a situation where it can only generate samples that satisfy a single data distribution. In this case, the discriminator will only treat the single data distribution satisfied by the generator as the true distribution and will only classify images satisfying this single distribution as real images.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this application is to propose a method and system for removing reflections from workpiece surface images based on a diffusion model. This method can learn the conditional mixing distribution during the reverse denoising process to generate a reflection-free image of the workpiece surface, thereby improving the accuracy of reflection removal from workpiece surface images.

[0006] To achieve the above objectives, one aspect of this application proposes a method for removing reflections from workpiece surface images based on a diffusion model, the method comprising:

[0007] An industrial camera is used to capture images of the workpiece to be processed and preprocess the data to obtain surface data images of the workpiece to be processed. The surface data images of the workpiece to be processed include non-reflective images and reflective images of the workpiece surface.

[0008] Based on the denoising diffusion probability model, a condition-guided branching structure is introduced to construct a workpiece image reflection removal network;

[0009] The surface data image of the workpiece to be processed is input into the workpiece image reflection removal network for reflection removal, and the reflection removal result of the workpiece surface image is obtained.

[0010] In some embodiments, the step of capturing and preprocessing the data of the workpiece to be processed using an industrial camera to obtain a surface data image of the workpiece includes:

[0011] Constructing a workpiece surface image acquisition device and acquiring the workpiece to be processed, wherein the workpiece surface image acquisition device includes an industrial camera, an industrial lens, a ring light source, and an industrial camera bracket;

[0012] By changing the shooting parameters of the workpiece surface image acquisition device, the workpiece to be processed is photographed and processed by the industrial camera to obtain a preliminary surface data image of the workpiece to be processed.

[0013] The preliminary surface data image of the workpiece to be processed is subjected to image format conversion processing to obtain the converted surface data image of the workpiece to be processed.

[0014] The converted surface data image of the workpiece to be processed is subjected to data augmentation processing to obtain the surface data image of the workpiece to be processed.

[0015] In some embodiments, changing the shooting parameters of the workpiece surface image acquisition device and shooting the workpiece to be processed by the industrial camera to obtain a preliminary surface data image of the workpiece to be processed includes:

[0016] The imaging parameters of the workpiece surface image acquisition device are obtained, including aperture coefficient, working distance and exposure time;

[0017] By fixing the aperture coefficient and changing the working distance and the exposure time, the workpiece to be processed is photographed and processed by the industrial camera to obtain a first surface data image of the workpiece to be processed.

[0018] By fixing the working distance, changing the aperture coefficient and the exposure time, and using the industrial camera to photograph and process the workpiece to be processed, a second surface data image of the workpiece to be processed is obtained.

[0019] By fixing the exposure time, changing the working distance and the aperture coefficient, and using the industrial camera to photograph and process the workpiece to be processed, a third surface data image of the workpiece to be processed is obtained.

[0020] By integrating the first surface data image of the workpiece to be processed, the second surface data image of the workpiece to be processed, and the third surface data image of the workpiece to be processed, the preliminary surface data image of the workpiece to be processed is obtained.

[0021] In some embodiments, the workpiece image reflection removal network includes a stitching structure, a condition-guided branching structure, and a noise prediction network.

[0022] In some embodiments, inputting the surface data image of the workpiece to be processed into a workpiece image reflection removal network for reflection removal to obtain the reflection removal result of the workpiece surface image includes:

[0023] The surface data image of the workpiece to be processed is input into the workpiece image reflection removal network;

[0024] Based on the forward diffusion stage of the workpiece image reflection removal network, Gaussian noise is gradually added to the non-reflective image of the workpiece surface according to variance scheduling to obtain a non-reflective image of the workpiece surface after noise addition.

[0025] The noise-added, non-reflective image of the workpiece surface and the reflective image of the workpiece surface are subjected to channel stitching to obtain a workpiece surface image data pair.

[0026] Based on the reverse denoising stage of the workpiece image reflection removal network, reflection removal is performed on the workpiece surface image data pairs to obtain the workpiece surface image reflection removal result.

[0027] In some embodiments, the expression for progressively adding Gaussian noise to the non-reflective image of the workpiece surface according to variance scheduling is specifically as follows:

[0028]

[0029] In the above formula, q(x) t |x t-1 ) represents the data distribution of the non-reflective image of the workpiece surface, t represents the noise level, and β represents the noise level. t This represents variance scheduling, where x0 represents a non-reflective image of the workpiece surface, and x... t This represents the image of the workpiece surface without reflection after adding noise during the t-diffusion step, x. t-1 The image represents the non-reflective surface of the workpiece after adding noise in the t-1 diffusion step, where T represents the total diffusion steps, and q(x) represents the total diffusion steps. 1:T |x0) represents the posterior estimated distribution of the non-reflective image of the workpiece surface during the forward diffusion stage. Let I represent the standard normal distribution, and let I represent the standard deviation.

[0030] In some embodiments, the inverse denoising stage based on the workpiece image reflection removal network, performing reflection removal on the workpiece surface image data pairs to obtain the workpiece surface image reflection removal result includes:

[0031] Based on the condition-guided branch structure of the workpiece image reflection removal network, the workpiece surface reflection image is subjected to reverse learning denoising processing to obtain constraint conditions.

[0032] Based on the noise prediction network of the workpiece image reflection removal network, noise prediction is performed on the workpiece surface image data pairs to obtain a noise vector.

[0033] By combining the constraints and the noise vector, the reflection image of the workpiece surface is subjected to reflection removal to obtain the reflection removal result of the workpiece surface image.

[0034] In some embodiments, the expression for the inverse denoising stage of the workpiece image reflection removal network is specifically as follows:

[0035]

[0036] In the above formula, x t This represents the image of the workpiece surface without reflection after adding noise during the t-diffusion step, x. t-1 The image represents the non-reflective surface of the workpiece after adding noise during the t-1 diffusion step, where T represents the total diffusion steps, and μ... θ (x t ,t) represents the mean, ∑ θ (x t ,t) denotes the covariance matrix, p θ (x 0:T p(x) represents the posterior estimated distribution of the non-reflective image of the workpiece surface during the reverse denoising stage. T ) represents the image data distribution at time T in the inverse denoising stage, p θ (x t-1 |x t () represents the posterior estimated distribution of the Gaussian noise added to the image at time t-1 during the reverse denoising stage, x T This represents the image of the workpiece surface at time T where there is no reflection. This represents a standard normal distribution.

[0037] In some embodiments, the expression for the reverse learning denoising process is specifically as follows:

[0038]

[0039] In the above formula, xt This represents the image of the workpiece surface without reflection after adding noise during the t-diffusion step, x. t-1 The image represents the non-reflective surface of the workpiece after adding noise in the t-1 diffusion step, where T represents the total number of diffusion steps. p(x) represents the reflective image of the workpiece surface. T () represents the image data distribution at time T in the inverse denoising stage, x T This represents the image of the workpiece surface at time T where there is no reflection. This indicates that the reverse denoising stage calculates the posterior estimated distribution of the non-reflective image of the workpiece surface based on the reflective image of the workpiece surface. This represents the posterior estimated distribution of the Gaussian noise added to the image at time t-1 by removing the reflective image of the workpiece surface during the reverse denoising stage.

[0040] To achieve the above objectives, another aspect of this application proposes a workpiece surface image reflection removal system based on a diffusion model, the system comprising:

[0041] The first module is used to take pictures of the workpiece to be processed and preprocess the data using an industrial camera to obtain surface data images of the workpiece to be processed, including non-reflective images and reflective images of the workpiece surface.

[0042] The second module is used to construct a reflection removal network for workpiece images based on a denoising diffusion probability model and by introducing a condition-guided branch structure.

[0043] The third module is used to input the surface data image of the workpiece to be processed into the workpiece image reflection removal network for reflection removal, and obtain the reflection removal result of the workpiece surface image.

[0044] The embodiments of this application include at least the following beneficial effects: This application provides a method and system for removing reflections from workpiece surface images based on a diffusion model. This scheme obtains a surface data image of the workpiece by taking pictures and preprocessing the data of the workpiece with an industrial camera. Then, based on a denoising diffusion probability model, a conditional guided branch structure is introduced to construct a workpiece image reflection removal network for reflection removal processing. By introducing a conditional guided branch structure, a workpiece surface reflection image is added as a condition on the diffusion model, guiding the model to learn the conditional mixture distribution during the reverse denoising process to generate a reflection-free image of the workpiece surface. This avoids the problems of pattern collapse and poor quality of the generated reflection-free image, which is prone to befalling in generative adversarial networks, and the easy retention of pixel edge information, thereby improving the accuracy of reflection removal from workpiece surface images. Attached Figure Description

[0045] Figure 1 This is a flowchart of a method for removing reflections from a workpiece surface image based on a diffusion model, provided in an embodiment of this application.

[0046] Figure 2 This is a schematic diagram of a workpiece surface image reflection removal system based on a diffusion model, provided in an embodiment of this application.

[0047] Figure 3 This is a schematic diagram of the structure of the workpiece surface image acquisition device provided in the embodiments of this application;

[0048] Figure 4 This is a schematic diagram of the structure of the workpiece image reflection removal network provided in the embodiments of this application;

[0049] Figure 5 This is a schematic diagram of the conditional branching structure provided in the embodiments of this application;

[0050] Figure 6 This is a schematic diagram of the reflective image of the workpiece surface provided in the embodiments of this application;

[0051] Figure 7 This is a schematic diagram of the reflection removal result of the workpiece surface image provided in the embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0053] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0054] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0056] Reference Figure 1 , Figure 1 A flowchart illustrating a method for removing reflections from workpiece surface images based on a diffusion model, as provided in this embodiment of the invention, is shown below. Figure 1 , the method comprises the following steps:

[0057] S100. The workpiece to be processed is photographed and preprocessed using an industrial camera to obtain surface data images of the workpiece to be processed, including non-reflective images and reflective images of the workpiece surface.

[0058] It should be noted that in some embodiments, step S100 may include: S110, constructing a workpiece surface image acquisition device and acquiring the workpiece to be processed. The workpiece surface image acquisition device includes an industrial camera, an industrial lens, a ring light source and an industrial camera bracket.

[0059] Specifically, if Figure 3 As shown, prepare the workpiece, industrial camera, industrial lens, standard light source, and industrial camera bracket. In the industrial production environment, place the workpiece, assemble the industrial lens and industrial camera, and mount them together with the standard light source on the industrial camera bracket to complete the hardware setup.

[0060] S120. Change the shooting parameters of the workpiece surface image acquisition device, and use an industrial camera to shoot and process the workpiece to be processed to obtain a preliminary surface data image of the workpiece to be processed.

[0061] Step S120 further includes acquiring the shooting parameters of the workpiece surface image acquisition device, including aperture coefficient, working distance, and exposure time; fixing the aperture coefficient, changing the working distance and exposure time, and shooting the workpiece to be processed with an industrial camera to obtain a first workpiece surface data image; fixing the working distance, changing the aperture coefficient and exposure time, and shooting the workpiece to be processed with an industrial camera to obtain a second workpiece surface data image; fixing the exposure time, changing the working distance and aperture coefficient, and shooting the workpiece to be processed with an industrial camera to obtain a third workpiece surface data image; integrating the first, second, and third workpiece surface data images to obtain a preliminary workpiece surface data image.

[0062] Specifically, under different lighting conditions, corresponding shooting parameters are selected to acquire reflective and non-reflective images of the workpiece surface, forming paired data pairs required for model training. The changed shooting parameters involve aperture, working distance, and exposure time. Three shooting schemes are specifically implemented. It should be noted that the main purpose of distinguishing between these three methods is to encompass as many commonly used shooting operations as possible in industrial scenarios, such as adjusting aperture, changing working distance, and adjusting exposure time. This allows for the verification of these three variables through controlled variables to support the universality of the embodiments of this invention.

[0063] S130. Perform image format conversion processing on the preliminary surface data image of the workpiece to be processed to obtain the converted surface data image of the workpiece to be processed.

[0064] Specifically, in this embodiment, since the images acquired in step S120 are stored in the camera's original RAW format, in order to facilitate subsequent data training, this embodiment converts the RAW format files into the common image file format BMP files for storage.

[0065] S140. Perform data augmentation processing on the converted surface data image of the workpiece to be processed to obtain the surface data image of the workpiece to be processed.

[0066] In some specific embodiments, image enhancement methods such as random flipping, random cropping, image scaling, and data padding are used during model training to enrich the diversity of image data and facilitate subsequent improvement of the robustness of model training.

[0067] S200: Based on the denoising diffusion probability model, a condition-guided branching structure is introduced to construct a workpiece image reflection removal network;

[0068] It should be noted that in some embodiments, the workpiece image reflection removal network includes a stitching structure, a condition-guided branching structure, and a noise prediction network.

[0069] It should be noted that, for example Figure 4 As shown, the network structure in this embodiment of the invention mainly adds a conditional guided branch to the original Denoising Diffusion Probabilistic Models (DDPMs). The original DDPMs were not chosen because the Markov transform learned during their inverse denoising process has randomness and will generate images with different high-level semantic information, which is not suitable for scenarios where it is necessary to generate workpiece surface images with specific high-level semantic information for de-reflection.

[0070] S300: Input the surface data image of the workpiece to be processed into the workpiece image reflection removal network to remove reflection, and obtain the reflection removal result of the workpiece surface image.

[0071] It should be noted that in some embodiments, step S300 may include: S310, inputting the surface data image of the workpiece to be processed into the workpiece image reflection removal network;

[0072] S320. Based on the forward diffusion stage of the workpiece image reflection removal network, Gaussian noise is gradually added to the non-reflective image of the workpiece surface according to the variance scheduling to obtain the non-reflective image of the workpiece surface after adding noise.

[0073] Specifically, in this embodiment, the real label of the non-reflective image on the workpiece surface is defined as x0, and the reflective image on the workpiece surface is defined as... During the forward diffusion process in the network structure training phase, the non-reflective images of the workpiece surface are scheduled according to variance β. t Gaussian noise is gradually added until the added noise makes the workpiece surface reflective. The data distribution q(x) of the image is then processed. t |x t-1 The noise level t tends to follow a Gaussian noise distribution. Specifically, the noise level t is sinusoidally position-encoded, and then the noise level t with the encoded position information is fed into a linear layer of size {[128,512],[512,512]} to calculate the input vector representation of the noise level t. Subsequently, the dimensionality change of the noise level t in the prediction network is kept consistent with the number of channels output by each network module through the mapping of the linear layer.

[0074] The specific expression for progressively adding Gaussian noise to the non-reflective image of the workpiece surface based on variance scheduling is shown below:

[0075]

[0076] In the above formula, q(x) t |x t-1 ) represents the data distribution of the non-reflective image of the workpiece surface, t represents the noise level, and β represents the noise level. t This represents variance scheduling, where x0 represents a non-reflective image of the workpiece surface, and x... t This represents the image of the workpiece surface without reflection after adding noise during the t-diffusion step, x. t-1 The image represents the non-reflective surface of the workpiece after adding noise in the t-1 diffusion step, where T represents the total diffusion steps, and q(x) represents the total diffusion steps. 1:T |x0) represents the posterior estimated distribution of the non-reflective image of the workpiece surface during the forward diffusion stage. Let I represent the standard normal distribution, and let I represent the standard deviation.

[0077] S330. Perform channel stitching on the noise-added non-reflective image of the workpiece surface and the reflective image of the workpiece surface to obtain a workpiece surface image data pair.

[0078] S340. In the reverse denoising stage based on the workpiece image reflection removal network, reflection removal is performed on the workpiece surface image data pairs to obtain the workpiece surface image reflection removal result.

[0079] Step S340 further includes: performing reverse learning denoising on the reflective image of the workpiece surface based on the condition-guided branch structure of the workpiece image reflection removal network to obtain constraint conditions; performing noise prediction on the workpiece surface image data pairs based on the noise prediction network of the workpiece image reflection removal network to obtain noise vectors; and performing reflection removal on the reflective image of the workpiece surface by combining the constraint conditions and the noise vectors to obtain the reflection removal result of the workpiece surface image.

[0080] In some specific embodiments, the present invention adds conditional guided branches to the network structure, the structure of which is as follows: Figure 5 As shown. Adding a conditional guiding branch does not affect the forward diffusion process of the network structure. Similarly, a pre-set variance scheduling method is used to progressively add Gaussian noise to the non-reflective image of the workpiece surface within T diffusion steps until the distribution of the non-reflective image of the workpiece surface conforms to a standard Gaussian distribution. Unlike the original denoising diffusion probability model, in the reverse denoising process, this invention uses a reflective image of the workpiece surface as a conditional guiding branch, allowing the model to draw noise from the joint data distribution. Mid-sampled data pairs Will The conditional input constraint model, used as the reverse process, learns the difference between reflective and non-reflective images of the workpiece surface to learn the denoising process. Its expression is:

[0081]

[0082] In the above formula, x t This represents the image of the workpiece surface without reflection after adding noise during the t-diffusion step, x. t-1 The image represents the non-reflective surface of the workpiece after adding noise in the t-1 diffusion step, where T represents the total number of diffusion steps. p(x) represents the reflective image of the workpiece surface. T () represents the image data distribution at time T in the inverse denoising stage, x T This represents the image of the workpiece surface at time T where there is no reflection. This indicates that the reverse denoising stage calculates the posterior estimated distribution of the non-reflective image of the workpiece surface based on the reflective image of the workpiece surface. This represents the posterior estimated distribution of the Gaussian noise added to the image at time t-1 by removing the reflective image of the workpiece surface during the reverse denoising stage.

[0083] The reverse denoising process uses A noise prediction network is used to predict the noise added to the non-reflective image of the workpiece surface at each step of the forward diffusion process. The input to this noise prediction network is a six-dimensional input image, i.e., the noisy non-reflective image of the workpiece surface x. t and reflective images of the workpiece surface The noise prediction network, composed of channels stitched together, along with t, serves as input to the noise prediction network. This network learns the noise level t added in each diffusion step T, and generates a non-reflective image x of the workpiece surface with added noise level t in each diffusion step. t The image is gradually denoised by subtracting the noise vector predicted by the noise network at that moment. When the diffusion step T is completed, the final denoised workpiece surface is obtained. The network output is still a six-channel image. The first three channels are the guiding conditional reflective image, and the last three channels are the non-reflective image corresponding to this reflective image.

[0084] That is, by learning the inverse denoising process, the original data distribution is recovered from this noisy image that conforms to a Gaussian distribution, thus obtaining a clean, non-reflective image of the workpiece surface. The expression for this is:

[0085]

[0086] In the above formula, x t This represents the image of the workpiece surface without reflection after adding noise during the t-diffusion step, x. t-1 The image represents the non-reflective surface of the workpiece after adding noise during the t-1 diffusion step, where T represents the total diffusion steps, and μ... θ (x t ,t) represents the mean, ∑ θ (x t ,t) denotes the covariance matrix, p θ (x 0:T p(x) represents the posterior estimated distribution of the non-reflective image of the workpiece surface during the reverse denoising stage. T ) represents the image data distribution at time T in the inverse denoising stage, p θ (x t-1 |x t () represents the posterior estimated distribution of the Gaussian noise added to the image at time t-1 during the reverse denoising stage, x T This represents the image of the workpiece surface at time T where there is no reflection. This represents a standard normal distribution.

[0087] The reverse denoising process mainly estimates p through a noise prediction network. θ (x t-1 |x t The mean μ of the distribution θ (x t ,t) and covariance matrix ∑ θ (xt ,t).

[0088] In summary, the training process of the workpiece image reflection removal network in this embodiment of the invention is as follows:

[0089] 1) Input a pair of images: one of a non-reflective workpiece surface and one of a reflective workpiece surface. Record as...

[0090] 2) From the distribution of joint data Medium-sampled workpiece surface reflection image pairs

[0091] 3) Set the diffusion step T and the noise level t, where t∈{1,2,3,…,T};

[0092] 4) Set the noise sampled from a standard Gaussian distribution, ε t ~N(0,I);

[0093] 5) According to Perform gradient descent until the loss function converges;

[0094] 6) Output the parameters θ of the noise prediction network.

[0095] The testing process of the workpiece image reflection removal network in this embodiment of the invention is as follows:

[0096] 1) Input the reflection image of the workpiece surface Noise prediction network Number of sampling steps S;

[0097] 2) Sample noise x from a standard Gaussian distribution t ;

[0098] 3) Within the sampling steps i = S,...,1, according to t = (i-1)·T / S+1, t next =t=(i-2)·T / S+1 Calculate x t , whose expression is:

[0099]

[0100] 4) Output a denoised image of the workpiece surface without reflection.

[0101] Finally, during the model training phase, this invention uses training data pairs consisting of reflective images and non-reflective images of the workpiece surface, and performs data format processing and image enhancement on all training data pairs. However, during model inference, only the reflective image of the workpiece surface needs to be input, such as... Figure 6 As shown, the conditional mixture distribution learned by the model during the training phase is used to predict the location of reflective areas in the workpiece surface image, and a non-reflective image of the workpiece surface is reconstructed by denoising. Figure 7As shown.

[0102] In summary, the embodiments of the present invention have the following beneficial effects compared with related technologies:

[0103] 1) Solve the problem in the industry that only highlights in workpiece surface images can be removed, but specular reflections in workpiece surface images cannot be removed.

[0104] 2) This addresses the problem that generative adversarial networks, commonly used in image specular reflection removal methods, are prone to pattern collapse, resulting in poor image quality and residual pixel edge information.

[0105] 3) Solve the problem that traditional denoising diffusion probability models generate unstable results and are prone to generating images with different high-level semantic information during the back-inference stage.

[0106] Please see Figure 2 This application also provides a workpiece surface image reflection removal system based on a diffusion model, which can implement the above-mentioned workpiece surface image reflection removal method based on a diffusion model. The system includes:

[0107] The first module 201 is used to take pictures of the workpiece to be processed and preprocess the data using an industrial camera to obtain surface data images of the workpiece to be processed, including non-reflective images and reflective images of the workpiece surface.

[0108] The second module 202 is used to construct a workpiece image reflection removal network based on a denoising diffusion probability model and by introducing a condition-guided branch structure.

[0109] The third module 203 is used to input the surface data image of the workpiece to be processed into the workpiece image reflection removal network for reflection removal, and obtain the reflection removal result of the workpiece surface image.

[0110] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0111] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for removing reflection from a workpiece surface image based on a diffusion model, characterized in that: The method comprises the following steps: The workpiece to be processed is photographed and data preprocessed by an industrial camera to obtain a surface data image of the workpiece to be processed, wherein the surface data image of the workpiece to be processed includes a non-reflective image of the workpiece surface and a reflective image of the workpiece surface; Based on the denoising diffusion probability model, a conditional guided branch structure is introduced to construct a workpiece image reflection removal network; The workpiece surface data image to be processed is input into the workpiece image reflection removal network to remove the reflection, and obtain the workpiece surface image reflection removal result.

2. The method according to claim 1, characterized in that The process of photographing and preprocessing the workpiece to be processed by an industrial camera to obtain a surface data image of the workpiece to be processed includes: Constructing a workpiece surface image acquisition device and acquiring a workpiece to be processed, wherein the workpiece surface image acquisition device includes an industrial camera, an industrial lens, a ring light source and an industrial camera bracket; Changing the shooting parameters of the workpiece surface image acquisition device, photographing the workpiece to be processed by the industrial camera, and obtaining a preliminary surface data image of the workpiece to be processed; Performing image format conversion processing on the preliminary surface data image of the workpiece to be processed to obtain a converted surface data image of the workpiece to be processed; The converted surface data image of the workpiece to be processed is subjected to data enhancement processing to obtain the surface data image of the workpiece to be processed.

3. The method according to claim 2, characterized in that The step of changing the shooting parameters of the workpiece surface image acquisition device, shooting the workpiece to be processed by the industrial camera, and obtaining a preliminary data image of the surface of the workpiece to be processed includes: Acquire shooting parameters of the workpiece surface image acquisition device, wherein the shooting parameters include aperture factor, working distance and exposure time; The aperture coefficient is fixed, the working distance and the exposure time are changed, and the workpiece to be processed is photographed by the industrial camera to obtain a first surface data image of the workpiece to be processed; The working distance is fixed, the aperture coefficient and the exposure time are changed, and the workpiece to be processed is photographed by the industrial camera to obtain a second surface data image of the workpiece to be processed; The exposure time is fixed, the working distance and the aperture coefficient are changed, and the workpiece to be processed is photographed by the industrial camera to obtain a third surface data image of the workpiece to be processed; The first data image of the surface of the workpiece to be processed, the second data image of the surface of the workpiece to be processed and the third data image of the surface of the workpiece to be processed are integrated to obtain the preliminary data image of the surface of the workpiece to be processed.

4. The method according to claim 1, characterized in that: The workpiece image reflection removal network includes a splicing structure, a conditional guided branch structure and a noise prediction network.

5. The method according to claim 1, characterized in that The step of inputting the workpiece surface data image to be processed into the workpiece image reflection removal network to remove the reflection, and obtaining the workpiece surface image reflection removal result, comprises: Inputting the surface data image of the workpiece to be processed into the workpiece image reflection removal network; Based on the forward diffusion stage of the workpiece image reflection removal network, Gaussian noise is gradually added to the workpiece surface non-reflection image according to variance scheduling to obtain the workpiece surface non-reflection image after adding noise; Perform channel stitching processing on the workpiece surface non-reflective image after adding noise and the workpiece surface reflective image to obtain a workpiece surface image data pair; Based on the reverse denoising stage of the workpiece image reflection removal network, the workpiece surface image data pair is subjected to reflection removal to obtain the workpiece surface image reflection removal result.

6. The method according to claim 5, characterized in that The expression for gradually adding Gaussian noise to the non-reflective image of the workpiece surface according to variance scheduling is specifically as follows: In the above formula, q(x t |x t-1 ) represents the data distribution of the image without reflection on the workpiece surface, t represents the noise level, β t represents variance scheduling, x0 represents the image without reflective surface of the workpiece, x t represents the non-reflective image of the workpiece surface after adding noise at the t diffusion step, x t-1 represents the non-reflective image of the workpiece surface after adding noise in the t-1 diffusion step, T represents the total diffusion step, q(x 1:T |x0) represents the posterior estimated distribution of the non-reflective image of the workpiece surface in the forward diffusion stage, represents the standard normal distribution and I represents the standard deviation.

7. The method according to claim 5, characterized in that The reverse denoising stage based on the workpiece image reflection removal network removes reflections from the workpiece surface image data pair to obtain the workpiece surface image reflection removal result, including: Based on the condition-guided branch structure of the workpiece image reflection removal network, reverse learning denoising is performed on the workpiece surface reflection image to obtain constraint conditions; Based on the noise prediction network of the workpiece image reflection removal network, noise prediction is performed on the workpiece surface image data pair to obtain a noise vector; The constraint condition and the noise vector are combined to remove the reflection of the reflective image on the workpiece surface to obtain a result of removing the reflection of the workpiece surface image.

8. The method according to claim 5, characterized in that The expression of the reverse denoising stage of the workpiece image reflection removal network is specifically as follows: In the above formula, x t represents the non-reflective image of the workpiece surface after adding noise at the t diffusion step, x t-1 represents the non-reflective image of the workpiece surface after adding noise in the t-1 diffusion step, T represents the total diffusion step, μ θ (x t ,t) represents the mean, ∑ θ (x t ,t) represents the covariance matrix, p θ (x 0:T ) represents the posterior estimated distribution of the non-reflective image of the workpiece surface during the reverse denoising stage, p(x T ) represents the image data distribution at time T in the reverse denoising stage, p θ (x t-1 |x t ) represents the posterior estimated distribution of the Gaussian noise added to the image at time t-1 in the reverse denoising stage, x T Indicates that there is no reflective image on the workpiece surface at time T. represents the standard normal distribution.

9. The method according to claim 7, characterized in that: The expression of the reverse learning denoising process is specifically as follows: In the above formula, x t represents the non-reflective image of the workpiece surface after adding noise at the t diffusion step, x t-1 represents the non-reflective image of the workpiece surface after adding noise in the t-1 diffusion step, T represents the total diffusion step, represents the reflected image of the workpiece surface, p(x T ) represents the image data distribution at time T in the reverse denoising stage, x T Indicates that there is no reflective image on the workpiece surface at time T. It indicates that the posterior estimated distribution of the non-reflective image of the workpiece surface is calculated based on the reflective image of the workpiece surface in the reverse denoising stage. It represents the posterior estimated distribution of the Gaussian noise added to the image removed at time t-1 according to the reflective image of the workpiece surface in the reverse denoising stage.

10. A workpiece surface image reflection removal system based on a diffusion model, characterized in that: The system comprises: The first module is used to photograph and pre-process the workpiece to be processed by an industrial camera to obtain a surface data image of the workpiece to be processed, wherein the surface data image of the workpiece to be processed includes a non-reflective image of the workpiece surface and a reflective image of the workpiece surface; The second module is used to introduce a conditional guided branch structure based on the denoising diffusion probability model to build a workpiece image reflection removal network; The third module is used to input the workpiece surface data image to be processed into the workpiece image reflection removal network to remove the reflection, and obtain the workpiece surface image reflection removal result.

Images