A part appearance defect detection method, device, equipment and storage medium

Abstract

The application discloses a part appearance defect detection method, device and equipment and a storage medium, and comprises the following steps: acquiring a plurality of images of a part to be detected under different illumination conditions through cameras arranged at preset positions, the number of the cameras being arranged so that the part to be detected has no dead angle in shooting, and each camera being arranged at a different position; preprocessing the images of the part to be detected, mixing the images of the part to be detected captured by each camera to obtain a pseudo-color image of the part to be detected under a preset angle; inputting the pseudo-color image into a preset defect detector, the defect detector outputting an identification result, and classifying the part to be detected according to the identification result. The application realizes omnibearing and dead-angle-free photometric stereo imaging, and improves the recall rate and stability of the detection algorithm by collecting more information about the surface depth and normal direction of the metal part.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of part detection, in particular to a part appearance defect detection method, device, equipment and storage medium. BACKGROUND

[0002] Appearance detection refers to the testing or measuring activities conducted to determine whether a product meets the design specified quality standards and technical requirements. Traditional appearance detection mainly relies on manpower, which has problems such as high work intensity, high repeatability, easy fatigue after long time work, and thus increases subjective errors.

[0003] At present, most of the existing technical solutions are for detecting two-dimensional planar materials, and the illumination scheme mostly uses coaxial light sources. The research in the field of surface defect detection mostly targets planar or continuous curved surface backgrounds, and defects are often clearly visible only under a specific illumination angle. At the same time, due to the convexity or concavity of the part shape, a single position camera cannot avoid missing the overall structure of the part. Therefore, multiple cameras are usually used in cooperation or the part is rotated and moved multiple times for shooting. This kind of 2D imaging method lacks micro feature information such as depth, normal direction and curvature when detecting metal surfaces with concave-convex structures, and it is difficult to detect and miss small defects such as pores and slag. For pseudodefects such as oxide skin and water stains and defects with depth information such as cracks and roll marks, the characteristics are very similar on the 2D gray image, and it is easy to confuse. In the production process, the proportion of non-defects is often much higher than that of real defects, which brings a large number of false detection signals and significantly reduces the practicability of the surface defect detection system. At present, there is no mature surface defect detection scheme for metal parts with complex geometric structures. Therefore, the present application provides a part appearance defect detection method, device, equipment and storage medium to solve the above problems. SUMMARY

[0004] In order to solve the above problems, the purpose of the present application is to provide a part appearance defect detection method, device, equipment and storage medium, which realizes omnidirectional and dead angle-free photometric stereo imaging, and improves the recall rate and stability of the detection algorithm by collecting more information such as the depth and normal direction of the metal part surface.

[0005] Based on this, the present application provides a part appearance defect detection method, which comprises:

[0006] A plurality of images of the part to be detected under different lighting conditions are acquired by cameras arranged at preset positions, the number of cameras is arranged so that the part to be detected has no dead angle for shooting, and each camera is arranged at a different position;

[0007] The image of the part to be detected is preprocessed, and the image of the part to be detected captured by each camera is mixed to obtain a pseudo-color image of the part to be detected at a preset angle.

[0008] The pseudo-color image is input into a preset defect detector, and the defect detector outputs an identification result, and the part to be detected is graded according to the identification result.

[0009] The preprocessing of the image of the part to be detected includes:

[0010] The image of the part to be detected is cropped, specifically including:

[0011] The image of the part to be detected is segmented into fixed-size slices, and adjacent segmented slices are kept with a preset percentage of overlapping areas, and the preset percentage is 15%.

[0012] The image of the part to be detected captured by each camera is mixed to obtain a pseudo-color image of the part to be detected at a preset angle, including:

[0013] The n images of the same size captured by each camera are mixed into one three-channel image according to a mixing algorithm.

[0014] The mixing algorithm includes:

[0015]

[0016] The mixed image is denoted as G, and the three-channel image is denoted as G1, G2, and G3, respectively. ij The mixing coefficient is a 3*n matrix, and the mixed image G can be displayed in RGB mode.

[0017] The defect detector includes:

[0018] The template matching module, the image classification module are connected in sequence.

[0019] The template matching module is used to extract the part contour in the pseudo-color image, and match the part contour with a preset defect-free part contour template to determine whether the part to be detected has contour missing.

[0020] The image classification module is used to classify the pseudo-color image without the contour missing, and the pseudo-color images belonging to the same defect are classified into one category, and the defect position, size, and category of the pseudo-color image are labeled by a label.

[0021] The method further comprises: randomly extracting the pseudo-color map input into the defect detector, and performing contrast enhancement on the pseudo-color map and receiving manual defect labeling of the pseudo-color map.

[0022] The method further comprises:

[0023] The manual defect labeling of the pseudo-color map is compared with the labeling of the pseudo-color map by the defect detector to determine the accuracy of the defect detector.

[0024] If the accuracy is lower than a preset accuracy threshold, the use of the defect detector is suspended, and the defect detector is retrained to improve defect detection accuracy.

[0025] The method further comprises: controlling the light-emitting condition of the light source by a PWM controller to realize that the part to be detected is in different lighting conditions.

[0026] The embodiment of the present application also provides a part appearance defect detection device, the device comprises:

[0027] The acquisition module is configured to acquire a plurality of images of the part to be detected under different lighting conditions by cameras arranged at preset positions, the number of cameras is arranged to ensure that the part to be detected has no dead angle, and each camera is arranged at a different position.

[0028] The processing and mixing module is configured to pre-process the images of the part to be detected, and mix the images of the part to be detected captured by each camera to obtain a pseudo-color map of the part to be detected under a preset angle.

[0029] The detection and grading module is configured to input the pseudo-color map into a preset defect detector, the defect detector outputs an identification result, and the part to be detected is graded according to the identification result.

[0030] The embodiment of the present application also provides a part appearance defect detection device, which comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the part appearance defect detection method when executing the computer program.

[0031] The embodiment of the present application also provides a computer readable storage medium, which comprises a stored computer program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to execute the part appearance defect detection method when the computer program runs.

[0032] In this invention, multiple light sources and cameras work together in a high-speed continuous shooting manner to achieve blind-spot-free photometric stereoscopic imaging through multi-angle illumination and multi-angle shooting. The advantage of this multi-camera arrangement is that it can capture multiple viewpoints simultaneously, eliminating factors that may cause occlusion and shadow problems under normal circumstances. By employing photometric stereoscopic illumination technology, different lighting methods are used in each frame, capturing the same scene under different lighting conditions, thereby increasing the information collection on the target object, i.e., the part to be inspected. Preprocessing the image of the part to be inspected, including cropping and image enhancement, helps the model capture more pixels to identify the features of minute defects and reduces computational load. The main function of the defect detector is to identify defects from the feature map and classify or grade them, labeling the location, size, edge, and category of defects in the image of the part to be inspected using tags. The methods of this invention can effectively improve the accuracy of defect detection. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of the part appearance defect detection method provided in the embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of cropping the image of the part to be detected according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of a part appearance defect detection device provided in an embodiment of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Figure 1 This is a flowchart of a part appearance defect detection method provided in an embodiment of the present invention, the method comprising:

[0039] S101. Obtain several images of the part to be detected under different lighting conditions through cameras arranged at preset positions, the number of cameras being arranged so that the part to be detected has no dead angle, and each camera being arranged at a different position;

[0040] The method further comprises: controlling the light-emitting condition of the light source through a PWM controller to realize that the part to be detected is under different lighting conditions.

[0041] Optionally, the transistor synchronization signal generator controls the light source and the industrial camera through a group of synchronization signals to continuously light and take pictures, and the industrial camera can be m and the light source can be k. These cameras and light sources are arranged in an upper and lower staggered manner around the detection target, i.e., the part to be detected, to ensure that each angle of the part to be detected can be photographed and illuminated to prevent dead angles.

[0042] The working principle is as follows:

[0043] The transistor synchronization signal generator sends N continuous frame sequences, denoted as 1-n, each frame illuminates the part to be detected by a plurality of light sources, and a specific industrial camera takes a picture at the same time. When all the frame sequences end, a total of m×n images can be obtained, which are respectively taken from m angles. Mixing the images of each view angle by different channels can obtain m-view photometric stereo images.

[0044] When all the frame sequences end, a total of m×n images can be obtained, which are respectively taken from m angles. Mixing the images of each view angle by different channels can obtain m-view photometric stereo images.

[0045] S102. Preprocess the images of the part to be detected, and mix the images of the part to be detected captured by each camera to obtain a pseudo-color image of the part to be detected under a preset angle;

[0046] The preprocessing of the images of the part to be detected comprises:

[0047] The images of the part to be detected are cropped, specifically including:

[0048] The images of the part to be detected are segmented into fixed-size slices, and adjacent segmented slices are kept with a preset percentage of overlapping areas, and the preset percentage is 15%. Figure 2 is a schematic diagram provided by the embodiment of the application for cropping the images of the part to be detected, please refer to Figure 2 .

[0049] The industrial surface defect detection task adopts a high-resolution camera for shooting, and the image resolution is as high as 2560*1920 pixels. The defect size in the metal part surface defect detection task is very small, and some defects are only 0.5 mm. Therefore, high-resolution images are helpful to capture more pixels to identify the characteristics of tiny defects. In order to process high-resolution images, the application adopts sliding cutting to divide the original picture into fixed-size slices. The adjacent slices after cutting retain a 15% overlap area to ensure that any area of defects can be completely detected. Note that these pretreatments are only to facilitate image processing by software and hardware, and are not the key steps of the application. Cutting the picture will increase hardware overhead, but does not affect the practicability of the application.

[0050] Wherein, the mixed images of each camera captured by the part to be detected are mixed to obtain a pseudo-color image of the part to be detected at a preset angle, including:

[0051] Each of the n images of the same size photographed by each camera is mixed into a three-channel image according to a mixing algorithm;

[0052] The mixing algorithm includes:

[0053]

[0054] Wherein, the mixed image is denoted as G, and the three-channel image is denoted as G1, G2 and G3, respectively. ij The mixing coefficient is denoted as a 3*n matrix θ, and the mixed image G can be displayed in RGB mode. Wherein, the numerical value of the parameter matrix θ is initialized as:

[0055]

[0056] The images of different light source illumination angles are orthogonal to each other in the RGB three channels. Here, the matrix form of θ is only a mathematical expression of the light source brightness parameter, and is not the only implementation way of the application. For example:

[0057] A deep convolutional network with n input channels, 3 output channels and a 1*1 convolution kernel size is used to realize image mixing, and θ is the weight matrix of the convolution kernel. The mixing operation is performed on the tensor to maintain the gradient flow, so that θ can be optimized through back propagation to improve the overall performance of defect detection.

[0058] S103. Input the pseudo-color image into a preset defect detector, and the defect detector outputs a recognition result, and grade the part to be detected according to the recognition result.

[0059] Wherein, the defect detector includes:

[0060] The template matching module, the image classification module are connected in sequence;

[0061] The template matching module is used for extracting a part contour in the pseudo-color image, and matching the part contour with a preset non-defective part contour template to determine whether the part to be detected has a contour defect.

[0062] The image classification module is used for classifying the pseudo-color image without the contour defect, classifying the pseudo-color images belonging to the same defect into one category, and labeling the defect position, size and category of the pseudo-color image through a label.

[0063] The template matching module is used for screening some large defects of the part to be detected, i.e. the contour defect. At this time, the part to be detected with the contour defect is a first-level defective product. The part to be detected without the contour defect needs to be further detected by the image classification module for some small defects such as pits and burrs.

[0064] The part with a preset first number range of small defects can be classified as a second-level defective product, and the part with a preset second number range of small defects can be classified as a third-level defective product. The minimum value of the second number range is greater than the maximum value of the first number range.

[0065] The image classification module can be internally provided with a defect library, and the defect library includes pseudo-color images with different small defects. The size of the small defect pseudo-color image is pre-set and smaller than the pseudo-color image of the part to be detected. A plurality of small defect pseudo-color images can be compared by moving a preset unit distance from top to bottom and from left to right to traverse the pseudo-color image of the part to be detected. When the comparison is consistent, the part to be detected has the defect. One part to be detected can also have multiple defects.

[0066] The defect result can be slag inclusion, pit, burr, etc.

[0067] The method further comprises: randomly extracting the pseudo-color image input into the defect detector, and performing contrast enhancement on the pseudo-color image and receiving manual defect labeling of the pseudo-color image.

[0068] The manual further defect labeling of the pseudo-color image is used to verify the accuracy of the defect detector.

[0069] The method further comprises:

[0070] The manual defect labeling of the pseudo-color image is compared with the labeling of the pseudo-color image by the defect detector to determine the accuracy of the defect detector.

[0071] If the accuracy is lower than a preset accuracy threshold, use of the defect detector is suspended and the defect detector is retrained to improve defect detection accuracy.

[0072] The defect detector can include models for target detection such as YOLO, FCOS, RT-DETR, CASCADE-RCNN, and improved versions thereof.

[0073] In the present application, multiple light sources and multiple cameras work together in high-speed continuous shooting mode to achieve multi-angle lighting and multi-angle shooting without dead angles. The advantage of this multi-camera arrangement is that it can capture multiple perspectives simultaneously, eliminating factors that can cause occlusion and shadow problems under normal circumstances. By using photometric stereo lighting technology, different lighting methods are used in each frame of shooting, and the same scene is shot under different lighting conditions, thereby increasing the collection of information about the target object, i.e., the part to be detected. Preprocessing of the part-to-be-detected image, i.e., cutting and image enhancement of the part-to-be-detected image, can help the model capture more pixels to identify the features of minor defects and reduce computational load. The main function of the defect detector is to identify defects from feature maps and classify or grade them, and to label the location, size, edge, and category of defects in the part-to-be-detected image. The method of the present application can effectively improve the accuracy of defect detection.

[0074] Figure 3 is a schematic diagram of a part appearance defect detection device provided by an embodiment of the present application, which comprises:

[0075] The acquisition module 301 is configured to acquire a plurality of images of the part to be detected under different lighting conditions through cameras arranged at preset positions, the number of cameras being arranged such that the part to be detected has no dead angle for shooting, and each camera being arranged at a different position.

[0076] The processing and mixing module 302 is configured to preprocess the images of the part to be detected and mix the images of the part to be detected captured by each camera to obtain a pseudo-color image of the part to be detected under a preset angle.

[0077] The detection and grading module 303 is configured to input the pseudo-color image into a preset defect detector, the defect detector outputs a recognition result, and the part to be detected is graded according to the recognition result.

[0078] The technical features and technical effects of the part appearance defect detection device proposed in the embodiment of the present application are the same as those of the method proposed in the embodiment of the present application, and will not be repeated here.

[0079] The embodiment of the present application also provides a part appearance defect detection device, comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the part appearance defect detection method when the computer program is executed.

[0080] The embodiment of the present application also provides a computer readable storage medium comprising a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute the part appearance defect detection method when the computer program runs.

[0081] In the present application, multiple light sources and multiple cameras work together in a high-speed continuous shooting mode to realize multi-angle lighting and multi-angle shooting without dead angle photometric stereo imaging. The advantage of this multi-camera arrangement is that it can capture multiple perspectives at the same time, eliminating factors that may cause occlusion and shadow problems under normal circumstances. By using photometric stereo lighting technology, different lighting methods are used in each frame of shooting, and the same scene is shot under different lighting conditions, thereby increasing the collection of information about the target object, i.e., the part to be detected. Preprocessing of the part to be detected image, i.e., cutting and image enhancement of the part to be detected image, can help the model capture more pixels to identify the features of minor defects and reduce the amount of calculation. The main function of the defect detector is to identify defects from the feature map and classify or grade the defects, and the position, size, edge and category of the defects in the part to be detected image are labeled by tags. The method of the present application can effectively improve the accuracy of defect detection.

[0082] The above only describes the preferred embodiments of the present application, and it should be noted that for those skilled in the art, without departing from the technical principles of the present application, several improvements and replacements can be made, and these improvements and replacements should also be considered as the protection scope of the present application.

Claims

1. A method of detecting a defect in the appearance of a part, characterized in that, The method comprises the following steps: A plurality of images of the part to be detected under different lighting conditions are acquired by cameras arranged at preset positions, the number of the cameras is arranged so that the part to be detected has no dead angle, each camera is arranged at a different position, and the part to be detected is under different lighting conditions by controlling the light-emitting state of the light source through a PWM controller, a transistor synchronization signal generator controls the light source and the industrial camera through a group of synchronization signals, the transistor synchronization signal generator sends N continuous frame sequences, denoted as 1~n, each frame is irradiated by a plurality of light sources to the part to be detected, and the part to be detected is photographed by the industrial camera, and different illumination modes are adopted in each frame; The images of the part to be detected are preprocessed, and the images of the part to be detected captured by each camera are mixed to obtain a pseudo-color image of the part to be detected under a preset angle, which comprises the following steps: The mixing algorithm comprises: Wherein, the mixed image is denoted as G, and the images of the three channels are respectively denoted as G1, G2 and G3, ij The mixing coefficients can be represented by a 3x n matrix, and the mixed image G is displayed in an RGB mode. The pseudo-color image is input into a preset defect detector, the defect detector outputs a recognition result, and the part to be detected is graded according to the recognition result.

2. The method of part appearance defect detection according to claim 1, wherein The preprocessing of the images of the part to be detected comprises: The images of the part to be detected are cropped, specifically comprising: The images of the part to be detected are segmented into fixed-size slices, and adjacent segmented slices are kept with a preset percentage of overlapping area, and the preset percentage is 15%.

3. The method of part appearance defect detection according to claim 1, wherein The defect detector comprises: a template matching module and an image classification module connected in sequence; The template matching module is used to extract the part contour in the pseudo-color image, and match the part contour with a preset defect-free part contour template to determine whether the part to be detected has contour missing; The image classification module is used to classify the pseudo-color image without the contour missing, the pseudo-color images belonging to the same defect are classified into one category, and the defect position, size and category of the pseudo-color image are labeled by a label.

4. The method of part appearance defect detection according to claim 1, wherein, The method further comprises: randomly extracting the pseudo-color image input into the defect detector, and performing contrast enhancement on the pseudo-color image and receiving manual defect labeling of the pseudo-color image.

5. The method of part appearance defect detection according to claim 4, wherein The method further comprises: The manual defect labeling of the pseudo-color image is compared with the labeling of the pseudo-color image by the defect detector to determine the accuracy of the defect detector; If the accuracy is lower than a preset accuracy threshold, the use of the defect detector is suspended, and the defect detector is retrained to improve the defect detection accuracy.

6. A part appearance defect detection apparatus characterized by comprising: The method comprises the following steps: The acquisition module is used to acquire a plurality of images of the part to be detected under different illumination conditions through cameras arranged at preset positions, the number of the cameras is arranged to make the part to be detected have no dead angle, each camera is arranged at a different position, and the light emitting condition of a light source is controlled through a PWM controller to make the part to be detected under different illumination conditions, a transistor synchronization signal generator controls the light source and the industrial camera through a group of synchronization signals, the transistor synchronization signal generator sends N continuous frame sequences, denoted as 1~n, each frame is irradiated by a plurality of light sources to the part to be detected, and is photographed by the industrial camera at the same time, and different illumination modes are adopted in each frame photographing; The processing and mixing module is used to pre-process the images of the part to be detected, and mixes the images of the part to be detected captured by each camera to obtain a pseudo-color image of the part to be detected under a preset angle, including: mixing n images of the same size photographed by each camera into a three-channel image according to a mixing algorithm; The mixing algorithm includes: Wherein, the mixed image is denoted as G, and the images of the three channels are respectively denoted as G1, G2 and G3, ij The mixing coefficients can be represented by a 3x n matrix, and the mixed image G is displayed in an RGB mode. The detection and grading module is used to input the pseudo-color image into a preset defect detector, the defect detector outputs an identification result, and grades the part to be detected according to the identification result. 7.A part appearance defect detection device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, when the computer program is executed by the processor, the part appearance defect detection method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer readable storage medium is located is controlled to execute the part appearance defect detection method according to any one of claims 1 to 5.

Images