A Surface Defect Detection Method and System Based on Multi-Light Source Collaboration

Abstract

This invention provides a surface defect detection method and system based on multi-light source collaboration. The method includes: acquiring a set of images of an object under illumination from light sources in different directions; randomly cropping and adding noise to the training set samples to obtain training data; inputting a three-channel RGB image to each branch to obtain multi-branch feature information; constructing a network and using max pooling to fuse the multi-branch feature information, aggregating a variable number of feature vectors into a feature map with a fixed number of channels, retaining the most significant features of each branch to obtain a detection model to be trained; using the training data as input to the detection model to be trained, completing the model's forward computation to obtain a trained detection model; and applying the trained detection model to a real-world industrial quality inspection scenario to detect surface defects on workpieces. This invention solves the problem of difficult detection and classification of three-dimensional defects in industrial quality inspection scenarios by restoring the three-dimensional shape of an object from a two-dimensional image.

Description

Technical Field

[0001] This invention relates to the technical field of industrial quality inspection, and more specifically, to a surface defect detection method and system based on multi-source collaborative detection. Background Technology

[0002] Due to limitations in manufacturing processes and transportation conditions, industrially produced parts often develop surface defects during production and transportation. To ensure the quality of products manufactured in automated processes, surface defect inspection of parts is an indispensable and crucial step. Previously, the inspection methods for these parts were typically divided into manual inspection and automated inspection.

[0003] A patent document with publication number CN112630230A discloses an online surface defect detection method based on photometric stereo method. During the detection process, the part moves continuously with the conveyor belt. When it reaches the detection area, the light sources distributed around the detection area light up in a cycle. At the same time, the camera simultaneously captures images under different lighting conditions. Throughout the process, the part moves continuously with the conveyor belt. During the capture of different images, the position of the part changes. The surface defect detection steps include: creating a template by extracting edge information, and achieving the unification of the part position in multiple images by matching the template to meet the prerequisites for subsequent photometric stereo method calculation; recovering the surface gradient and curvature information of the part under different workstations using photometric stereo method; extracting the surface defects of the part by means of binarization, edge detection, and connected component detection feature extraction in sequence, and performing preliminary screening based on thresholds to determine whether the processing quality of the part to be tested is qualified.

[0004] A photometric stereoscopic method and apparatus for detecting and analyzing surface defects of an object are disclosed in patent document CN113658155A. The method includes: acquiring pixel values ​​from images of the surface of the object under test from at least three different angle light sources to construct a photometric stereoscopic mathematical model; calculating the reflectance of the photometric stereoscopic mathematical model to construct a reflectance map; calculating the gradient field of the surface of the object under test; calculating the Gaussian surface curvature and the average surface curvature based on the gradient field of the surface to obtain a Gaussian surface curvature defect map and an average surface curvature defect map; and analyzing the reflectance map, the Gaussian surface curvature defect map, and the average surface curvature defect map to obtain the detection result.

[0005] Its disadvantages are: 1. Using convolutional neural networks requires the same number and size of input images for training and testing, hindering the flexible application of the model. 2. The direction of the light source must be used as input during training and testing, making it difficult to apply in scenarios where accurate light source information is unavailable. 3. Insufficient training set samples and unsupervised learning methods result in unsatisfactory 3D reconstruction results. 4. Directly modeling the reflectivity of non-Lambertian surfaces is computationally cumbersome and difficult to adapt to objects of different materials.

[0006] Therefore, a new technical solution is needed to improve the above-mentioned technical problems. Summary of the Invention

[0007] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a surface defect detection method and system based on multi-light source collaboration.

[0008] According to the present invention, a surface defect detection method based on multi-source collaborative processing is provided, the method comprising the following steps:

[0009] Step S1: Based on the basic principle of photometric stereo method, acquire a set of images of the object under illumination from different directions using the same camera viewpoint, and calculate the normal vector of the object surface based on the illumination intensity of each pixel.

[0010] Step S2: Using the simulation datasets Blobby Dataset and Sculpture Dataset as training sets, randomly crop the training set samples and add noise to obtain the training data;

[0011] Step S3: The fully convolutional neural network is a multi-branch Siamese network. At the input end, each branch network has the same configuration, parameters and weights. Each branch takes a three-channel RGB image as input to obtain multi-branch feature information.

[0012] Step S4: Build the network and use max pooling to fuse the multi-branch feature information output in step S3. Aggregate the feature vectors of varying number into a feature map with a fixed number of channels, retain the most significant features of each branch, and obtain the detection model to be trained.

[0013] Step S5: Using the training data obtained in step S2, input the detection model to be trained obtained in step S4, complete the forward calculation of the model, and obtain the trained detection model.

[0014] Step S6: Use the trained detection model obtained in step S5 in actual industrial quality inspection scenarios to detect surface defects on workpieces.

[0015] Preferably, in step S1, the camera is fixed vertically downward, and four light sources in different directions are set up; each light source illuminates the object to be tested in turn, and the camera takes pictures of the object illuminated by a single light source, and the collected images are used as test data.

[0016] Preferably, in step S3, each single branch uses the backbone of Darknet-53 for feature extraction and a feature pyramid structure for feature fusion. Darknet-53 performs downsampling on the input image five times to compress the image and reduce parameters. The feature pyramid structure fuses high-level feature information with low-level feature information from top to bottom and passes it to the next layer to enhance feature extraction and obtain multi-branch feature information.

[0017] Preferably, in step S4, the fused feature information is input into the normal vector regression network; the normal vector regression network is configured with two upsampling layers and one deconvolution layer; at the end of the normal vector regression network, an L2 normalization layer is used to unify the dimensions of the feature vector and generate a surface normal vector; after the network is built, the detection model to be trained is obtained.

[0018] Preferably, in step S5, the training result is supervised by the estimation error between the predicted normal vector and the true normal vector; the loss function is represented by cosine similarity; the difference between the actual output and the expected output is calculated through the loss function; the gradient of the learnable parameters in the model is obtained according to the gradient backpropagation algorithm; the gradient parameter update is completed according to the optimizer configuration; forward propagation is performed again, and the process is repeated iteratively; as the loss function gradually decreases, the training of the model is completed, and a trained detection model is obtained.

[0019] The present invention also provides a surface defect detection system based on multi-light source collaboration, the system comprising the following modules:

[0020] Module M1: Based on the basic principles of photometric stereo method, it acquires a set of images of an object under illumination from different directions using the same camera viewpoint, and calculates the normal vector of the object's surface based on the illumination intensity of each pixel.

[0021] Module M2: Uses the simulation datasets Blobby Dataset and Sculpture Dataset as training sets, randomly prunes the training set samples and adds noise to obtain the training data;

[0022] Module M3: The fully convolutional neural network is a multi-branch Siamese network. At the input end, each branch network has the same configuration, parameters and weights. Each branch takes a three-channel RGB image as input to obtain multi-branch feature information.

[0023] Module M4: The network is built by using max pooling to fuse the multi-branch feature information output by module M3, aggregating a variable number of feature vectors into a feature map with a fixed number of channels, retaining the most significant features of each branch, and obtaining the detection model to be trained.

[0024] Module M5: Using the training data obtained in Module M2, input the detection model to be trained obtained in Module M4, complete the forward calculation of the model, and obtain the trained detection model;

[0025] Module M6: The trained detection model obtained from Module M5 is used in actual industrial quality inspection scenarios to detect surface defects on workpieces.

[0026] Preferably, module M1 fixes the camera orientation vertically downward and sets four light sources in different directions; each light source illuminates the object to be tested in turn, and the camera takes pictures of the object illuminated by a single light source, and the collected images are used as test data.

[0027] Preferably, each single branch in module M3 uses the backbone of Darknet-53 for feature extraction and a feature pyramid structure for feature fusion. Darknet-53 performs downsampling on the input image five times to compress the image and reduce parameters. The feature pyramid structure fuses high-level feature information with low-level feature information from top to bottom and passes it to the next layer to enhance feature extraction and obtain multi-branch feature information.

[0028] Preferably, the fused feature information is input into the normal vector regression network in module M4; the normal vector regression network is configured with two upsampling layers and one deconvolution layer; at the end of the normal vector regression network, an L2 normalization layer is used to unify the dimensions of the feature vector and generate a surface normal vector; after the network is built, the detection model to be trained is obtained.

[0029] Preferably, the training results in module M5 are supervised by the estimation error between the predicted normal vector and the true normal vector; the loss function is represented by cosine similarity; the difference between the actual output and the expected output is calculated through the loss function; the gradient of the learnable parameters in the model is obtained according to the gradient backpropagation algorithm; the gradient parameter update is completed according to the optimizer configuration; forward propagation is performed again, and the process is repeated iteratively; as the loss function gradually decreases, the training of the model is completed, and a trained detection model is obtained.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. This invention proposes a surface defect detection method based on photometric stereo vision, which restores the three-dimensional shape of an object from a two-dimensional image, thus solving the problem of difficult detection and classification of three-dimensional defects in industrial quality inspection scenarios;

[0032] 2. This invention proposes a fully convolutional neural network based on multi-branch fusion, which solves the problems of classical photometric stereo methods being difficult to apply to real objects and non-Lambertian photometric stereo methods being cumbersome and complex to model by directly learning the mapping from the pixel intensity of the object surface to the surface normal map.

[0033] 3. This invention uses a multi-branch fusion fully convolutional neural network, and the number and size of input images can be arbitrarily set during training and testing, thus expanding the application space of the model.

[0034] 4. The model training and testing of this invention do not require the light source direction as input, realizing a non-calibrated photometric stereo method, which can be more flexibly applied to 3D reconstruction scenes where the light source direction is unknown;

[0035] 5. This invention is based on Blobby Dataset and Sculpture Dataset for training, and a simulated workpiece dataset is constructed to obtain sufficient training and testing data. Supervised learning is carried out by using the estimation error between the predicted normal vector and the true normal vector, and good testing results are achieved.

[0036] 6. This invention does not require modeling the reflectivity of the surface of the object under test, and directly realizes the mapping from the pixel intensity of the surface of the object under test to the surface normal map. Attached Figure Description

[0037] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0038] Figure 1 This is a diagram of the experimental apparatus for the present invention;

[0039] Figure 2 This is a flowchart of the defect detection process for the experimental apparatus of the present invention;

[0040] Figure 3 This is a diagram of the fully convolutional neural network structure based on multi-branch fusion in this invention;

[0041] Figure 4 This is a structural diagram of the multi-branch module of the present invention;

[0042] Figure 5 This is the training logic diagram for the present invention;

[0043] Figure 6 This is the inference logic diagram for the present invention. Detailed Implementation

[0044] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0045] Example 1:

[0046] According to the present invention, a surface defect detection method based on multi-source collaborative processing is provided, the method comprising the following steps:

[0047] Step S1: Based on the basic principle of photometric stereo method, acquire a set of images of the object under illumination from different directions using the same camera viewpoint, and calculate the normal vector of the object surface based on the illumination intensity of each pixel; fix the camera direction vertically downward, and set up four light sources in different directions; each light source illuminates the object under test in turn, and the camera takes pictures of the object under illumination from a single light source, and the acquired images are used as test data.

[0048] Step S2: Using the simulation datasets Blobby Dataset and Sculpture Dataset as training sets, randomly crop the training set samples and add noise to obtain the training data;

[0049] Step S3: The fully convolutional neural network is a multi-branch Siamese network. At the input end, each branch network has the same configuration, parameters, and weights. Each branch takes a three-channel RGB image as input to obtain multi-branch feature information. Each single branch uses the backbone of Darknet-53 for feature extraction and uses a feature pyramid structure for feature fusion. Darknet-53 downsamples the input image five times to compress the image and reduce parameters. The feature pyramid structure fuses high-level feature information with low-level feature information from top to bottom and passes it to the next layer to enhance feature extraction and obtain multi-branch feature information.

[0050] Step S4: Construct the network by using max pooling to fuse the multi-branch feature information output in step S3, aggregating a variable number of feature vectors into a feature map with a fixed number of channels, retaining the most significant features of each branch, and obtaining the detection model to be trained; input the fused feature information into the normal vector regression network; the normal vector regression network is configured with two upsampling layers and one deconvolution layer; at the end of the normal vector regression network, an L2 normalization layer is used to unify the dimensions of the feature vectors and generate surface normal vectors; after the network is constructed, the detection model to be trained is obtained.

[0051] Step S5: Using the training data obtained in Step S2, input the detection model to be trained obtained in Step S4 to complete the forward computation of the model and obtain the trained detection model; the training result is supervised by the estimation error between the predicted normal vector and the true normal vector; the loss function is represented by cosine similarity; the difference between the actual output and the expected output is calculated through the loss function, and the gradient of the learnable parameters in the model is obtained according to the gradient backpropagation algorithm. The gradient parameter update is completed according to the optimizer configuration, and forward propagation is performed again, iterating repeatedly; as the loss function gradually decreases, the training of the model is completed and the trained detection model is obtained.

[0052] Step S6: Use the trained detection model obtained in step S5 in actual industrial quality inspection scenarios to detect surface defects on workpieces.

[0053] Example 2:

[0054] Example 2 is a preferred embodiment of Example 1, and is used to illustrate the present invention in more detail.

[0055] The present invention also provides a surface defect detection system based on multi-light source collaboration, the system comprising the following modules:

[0056] Module M1: Based on the basic principles of photometric stereo method, a set of images of an object under illumination from different directions are acquired from the same camera perspective. The normal vector of the object surface is calculated based on the illumination intensity of each pixel. The camera direction is fixed vertically downward, and four light sources in different directions are set. The light sources in each direction illuminate the object under test in turn, and the camera takes pictures of the object under illumination from a single light source. The acquired images are used as test data.

[0057] Module M2: Uses the simulation datasets Blobby Dataset and Sculpture Dataset as training sets, randomly prunes the training set samples and adds noise to obtain the training data;

[0058] Module M3: The fully convolutional neural network is a multi-branch Siamese network. At the input end, each branch network has the same configuration, parameters, and weights. Each branch takes a three-channel RGB image as input to obtain multi-branch feature information. Each single branch uses the backbone of Darknet-53 for feature extraction and uses a feature pyramid structure for feature fusion. Darknet-53 downsamples the input image five times to compress the image and reduce parameters. The feature pyramid structure fuses high-level feature information with low-level feature information from top to bottom and passes it to the next layer to enhance feature extraction and obtain multi-branch feature information.

[0059] Module M4: The network is built by fusing the multi-branch feature information output by module M3 using max pooling. The variable number of feature vectors are aggregated into a feature map with a fixed number of channels, retaining the most significant features of each branch to obtain the detection model to be trained. The fused feature information is input into the normal vector regression network. The normal vector regression network has two upsampling layers and one deconvolution layer. At the end of the normal vector regression network, an L2 normalization layer is used to unify the dimensions of the feature vectors and generate surface normal vectors. After the network is built, the detection model to be trained is obtained.

[0060] Module M5: Using the training data obtained in Module M2, input the detection model to be trained obtained in Module M4 to complete the forward computation of the model and obtain the trained detection model; the training result is supervised by the estimation error between the predicted normal vector and the true normal vector; the loss function is represented by cosine similarity; the difference between the actual output and the expected output is calculated through the loss function, and the gradient of the learnable parameters in the model is obtained according to the gradient backpropagation algorithm. The gradient parameter update is completed according to the optimizer configuration, and forward propagation is performed again, iterating repeatedly; as the loss function gradually decreases, the training of the model is completed and the trained detection model is obtained.

[0061] Module M6: The trained detection model obtained from Module M5 is used in actual industrial quality inspection scenarios to detect surface defects on workpieces.

[0062] Example 3:

[0063] Example 3 is a preferred example of Example 1, and is used to illustrate the present invention in more detail.

[0064] This patent proposes a new surface defect detection method for industrial quality inspection scenarios. It uses photometric stereo method to reconstruct the three-dimensional surface of the object under test, thereby achieving high-precision perception of three-dimensional defects in the plane, such as indentations and bulges.

[0065] This patent proposes a fully convolutional neural network based on multi-branch fusion, which includes feature extraction, feature fusion, pooling layers, and a normal vector regression network, comprising the following modules:

[0066] Module 1, Experimental Apparatus Module. Based on the fundamental principles of photometric stereochemistry, a set of images of an object illuminated by light sources from different directions are acquired from the same camera perspective. The normal vector of the object's surface is calculated based on the illumination intensity of each pixel. This patent fixes the camera direction vertically downwards and sets up four light sources in different directions. Since this patent does not require learning the direction of the light sources, calibration of the light sources is unnecessary. In the image acquisition stage, the light sources from each direction illuminate the object under test in turn, and the camera takes pictures of the object under the illumination of each individual light source. The acquired images serve as test data.

[0067] Module Two: Training Data Processing Module. This patent uses the simulation datasets Blobby Dataset and Sculpture Dataset as the training set. To reduce the gap between the training set and the real scene, this patent randomly prunes the training set samples and adds noise to obtain the training data.

[0068] Module 3, Multi-branch Module. The fully convolutional neural network proposed in this patent is a multi-branch Siamese network. At the input end, each branch network has the same configuration, parameters, and weights, and each branch takes a three-channel RGB image as input. Each single branch uses the backbone of Darknet-53 for feature extraction and uses a feature pyramid structure for feature fusion. Darknet-53 downsamples the input image five times to compress the image and reduce parameters. The feature pyramid structure fuses high-level feature information with low-level feature information from top to bottom and then passes it to the next layer, enhancing feature extraction and obtaining high-resolution, semantically strong multi-branch feature information.

[0069] Module 4, Network Structure Module. The network is built by first fusing the multi-branch feature information output from the multi-branch modules using max pooling. This aggregates a variable number of feature vectors into a feature map with a fixed number of channels, preserving the most salient features from each branch. The fused feature information is then input into the normal vector regression network. This network incorporates two upsampling layers and one deconvolution layer to ensure the final output feature map maintains the same dimensionality as the input image. At the end of the normal vector regression network, an L2 normalization layer is used to unify the dimensions of the feature vectors and generate surface normal vectors. After the network is built, the detection model to be trained is obtained.

[0070] Module 5, Training Module. This patent uses the training data obtained in the data processing module as input to the detection model to be trained obtained in the network structure module to complete the forward computation of the model. The training result is supervised by the estimation error between the predicted normal vector and the true normal vector. This patent uses cosine similarity to represent the loss function. For vectors A and B, the cosine value of the angle between them can be expressed as:

[0071]

[0072] Therefore, the loss function of the fully convolutional neural network proposed in this patent can be expressed as:

[0073]

[0074] Where, N ij Let (i,j) represent the true unit normal vector at point (i,j). Let w represent the predicted unit normal vector at point (i,j), w be the width of the input image, and h be the height of the input image. If the predicted normal vector and the true normal vector are sufficiently similar, then... The value of will be close enough to 1 that the value of the loss function will be very small. If the predicted normal vector deviates too much from the true normal vector, then... The loss function value will approach 0, resulting in a large value. The difference between the actual and expected outputs is calculated using the loss function. Based on the gradient backpropagation algorithm, the gradients of the learnable parameters in the model are obtained. The parameters are updated according to the gradient based on the optimizer configuration, and forward propagation is performed again, iterating repeatedly. As the loss function gradually decreases, the model training is complete, resulting in a trained detection model.

[0075] Module Six, Inference Module. The trained detection model obtained from the training module is applied to actual industrial quality inspection scenarios to detect surface defects on workpieces. Test data is input into the trained detection model, and the model's normal vector regression network outputs a reconstructed surface normal map. This patent performs image processing and defect detection on the recovered surface normal map. During the surface normal map recovery process, a defect-free sample is input into the model to recover a defect-free surface normal map, which serves as a mask. The normal vector value of the corresponding image is subtracted from the normal vector value of the mask to highlight the normal information at the defect location. Subsequent defect detection is based on the defect normal vector value after removing the mask. Then, this patent performs grayscale conversion, Gaussian blurring, opening operation, and Canny edge detection on the image to eliminate noise and identify defect edges. Finally, detection boxes are generated by identifying defect edges, and multiple detection boxes of neighboring points are merged into one to achieve defect detection.

[0076] In industrial quality inspection environments, traditional two-dimensional defect detection strategies are prone to misclassification and false detections of defects such as indentations and bulges. This patent proposes a surface defect detection method based on photometric stereo vision, which reconstructs the three-dimensional shape of an object from a two-dimensional image, solving the problem of difficult detection and classification of three-dimensional defects in industrial quality inspection scenarios.

[0077] This patent proposes a fully convolutional neural network based on multi-branch fusion, which solves the problems of classical photometric stereo methods being difficult to apply to real objects and non-Lambertian photometric stereo methods being cumbersome and complex by directly learning the mapping from the pixel intensity of the object surface to the surface normal map.

[0078] Those skilled in the art can understand this embodiment as a more specific description of Embodiment 1 and Embodiment 2.

[0079] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0080] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A surface defect detection method based on multi-source collaborative processing, characterized in that, The method includes the following steps: Step S1: Based on the basic principle of photometric stereo method, acquire a set of images of the object under illumination from different directions using the same camera viewpoint, and calculate the normal vector of the object surface based on the illumination intensity of each pixel. Step S2: Using the simulation datasets Blobby Dataset and Sculpture Dataset as training sets, randomly crop the training set samples and add noise to obtain the training data; Step S3: The fully convolutional neural network is a multi-branch Siamese network. At the input end, each branch network has the same configuration, parameters and weights. Each branch takes a three-channel RGB image as input to obtain multi-branch feature information. Step S4: Build the network and use max pooling to fuse the multi-branch feature information output in step S3. Aggregate the feature vectors of varying number into a feature map with a fixed number of channels, retain the most significant features of each branch, and obtain the detection model to be trained. Step S5: Using the training data obtained in step S2, input the detection model to be trained obtained in step S4, complete the forward calculation of the model, and obtain the trained detection model. Step S6: Use the trained detection model obtained in step S5 in actual industrial quality inspection scenarios to detect surface defects on workpieces; In step S3, each single branch uses the backbone of Darknet-53 for feature extraction and a feature pyramid structure for feature fusion. Darknet-53 downsamples the input image five times to compress the image and reduce parameters. The feature pyramid structure fuses high-level feature information with low-level feature information from top to bottom and passes it to the next layer to enhance feature extraction and obtain multi-branch feature information.

2. The surface defect detection method based on multi-source collaborative processing according to claim 1, characterized in that, In step S1, the camera is fixed vertically downward, and four light sources in different directions are set up. Each light source illuminates the object to be tested in turn, and the camera takes pictures of the object illuminated by a single light source. The collected images are used as test data.

3. The surface defect detection method based on multi-source collaborative processing according to claim 1, characterized in that, In step S4, the fused feature information is input into the normal vector regression network; the normal vector regression network is configured with two upsampling layers and one deconvolution layer; at the end of the normal vector regression network, an L2 normalization layer is used to unify the dimensions of the feature vector and generate the surface normal vector; after the network is built, the detection model to be trained is obtained.

4. The surface defect detection method based on multi-source collaborative processing according to claim 1, characterized in that, The training results in step S5 are supervised by the estimation error between the predicted normal vector and the true normal vector; the loss function is represented by cosine similarity; the difference between the actual output and the expected output is calculated through the loss function; the gradient of the learnable parameters in the model is obtained according to the gradient backpropagation algorithm; the gradient parameter update is completed according to the optimizer configuration; forward propagation is performed again, and the process is repeated iteratively; as the loss function gradually decreases, the training of the model is completed, and the trained detection model is obtained.

5. A surface defect detection system based on multi-source collaborative operation, characterized in that, The system includes the following modules: Module M1: Based on the basic principles of photometric stereo method, it acquires a set of images of an object under illumination from different directions using the same camera viewpoint, and calculates the normal vector of the object's surface based on the illumination intensity of each pixel. Module M2: Uses the simulation datasets Blobby Dataset and Sculpture Dataset as training sets, randomly prunes the training set samples and adds noise to obtain the training data; Module M3: The fully convolutional neural network is a multi-branch Siamese network. At the input end, each branch network has the same configuration, parameters and weights. Each branch takes a three-channel RGB image as input to obtain multi-branch feature information. Module M4: The network is built by using max pooling to fuse the multi-branch feature information output by module M3, aggregating a variable number of feature vectors into a feature map with a fixed number of channels, retaining the most significant features of each branch, and obtaining the detection model to be trained. Module M5: Using the training data obtained in Module M2, input the detection model to be trained obtained in Module M4, complete the forward calculation of the model, and obtain the trained detection model; Module M6: The trained detection model obtained in Module M5 is used in actual industrial quality inspection scenarios to detect surface defects on workpieces; Each single branch in module M3 uses the backbone of Darknet-53 for feature extraction and a feature pyramid structure for feature fusion. Darknet-53 downsamples the input image five times to compress the image and reduce parameters. The feature pyramid structure fuses high-level feature information with low-level feature information from top to bottom and passes it to the next layer to enhance feature extraction and obtain multi-branch feature information.

6. The surface defect detection system based on multi-source collaborative operation according to claim 5, characterized in that, The module M1 fixes the camera orientation vertically downward and sets up four light sources in different directions; each light source illuminates the object under test in turn, and the camera takes pictures of the object under the illumination of a single light source, and the collected images are used as test data.

7. The surface defect detection system based on multi-source collaborative operation according to claim 5, characterized in that, In module M4, fused feature information is input into the normal vector regression network; the normal vector regression network is configured with two upsampling layers and one deconvolution layer; at the end of the normal vector regression network, an L2 normalization layer is used to unify the dimensions of the feature vector and generate surface normal vectors; after the network is built, the detection model to be trained is obtained.

8. The surface defect detection system based on multi-source collaborative operation according to claim 5, characterized in that, The training results in module M5 are supervised by the estimation error between the predicted normal vector and the true normal vector; the loss function is represented by cosine similarity; the difference between the actual output and the expected output is calculated through the loss function, and the gradient of the learnable parameters in the model is obtained according to the gradient backpropagation algorithm. The gradient parameter update is completed according to the optimizer configuration, and forward propagation is performed again, iterating repeatedly; as the loss function gradually decreases, the training of the model is completed, and the trained detection model is obtained.

Images