An apparatus and method for defect detection of a highly reflective machining surface

Abstract

The present invention relates to a device and method for defect detection of a highly reflective machined surface, belonging to the field of vision detection. The method includes a rotary table, an image acquisition unit, an arc surface light source and a control unit; the rotary table is used to accurately control the irradiation position of the object to be measured; the rotary table includes a stepping motor, a reducer, a coupling, a flange shaft, a transition plate, a tailstock, a motor base, a bottom plate, a support frame and a quick clamping device; the quick clamping device is composed of a tailstock, a center and a fixing pin; the fixing pin is connected to a locking plate and functions to position and transmit torque; the center cooperates with the tailstock to fix both ends of the object to be measured, meeting the requirement of quick clamping and being applicable to crankshafts of different lengths; the present invention proposes an illumination method of zoning light control for the arc surface light source, which has a simple structure and can simulate the irradiation of light sources at different angles, maximizing the retention of image features and facilitating subsequent image processing.

Description

Technical Field

[0001] The present invention belongs to the field of vision detection and relates to a device and method for detecting defects on a highly reflective machined surface. Background Art

[0002] In the mechanical processing automation industry, due to changes in the tool path during the processing process, the inherent properties of the material, vibration, tool damage, and improper polishing treatment, etc., defects such as dents, scratches, appearance deformation, incorrect orientation, and unsatisfactory reflective characteristics will be formed on the surface of mechanical processing parts. For highly reflective mechanical processing parts, due to the influence of the reflected light on the measured surface, it is difficult to identify various defect types and eliminate false defects (water film, shadow, oil stain, and non-depth vibration marks). In the method described in Patent CN-105849534B, by obtaining images after irradiating with multiple light sources from different angles and performing differential processing to obtain the defect positions, it can only be applied to regular cylindrical surfaces, and the robustness of defect segmentation is low, and it cannot be applied to small target defects. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a device and method for detecting defects on a highly reflective machined surface to achieve precise positioning and segmentation of defects on complex curved surfaces.

[0004] To achieve the above purpose, the present invention provides the following technical solutions:

[0005] A device for detecting defects on a highly reflective machined surface, the device includes a rotating table, an image acquisition unit, an arc surface light source, and a control unit;

[0006] The rotating table is used to precisely control the irradiation position of the object to be measured;

[0007] The rotating table includes a stepping motor, a reducer, a coupling, a flange shaft, a transition plate, a tailstock, a motor base, a bottom plate, a support frame, and a quick clamping device;

[0008] The quick clamping device is composed of a tailstock, a center, and a fixing pin;

[0009] The fixing pin is connected to the locking plate and plays a role in positioning and transmitting torque;

[0010] The center cooperates with the tailstock to fix both ends of the object to be measured, meeting the requirements of quick clamping and being applicable to crankshafts of different lengths;

[0011] The reducer adopts worm and worm gear reduction and is connected to one end of the quick clamping device through a coupling;

[0012] The arc surface light source is placed directly above the object to be measured, and the arc surface light source is used to irradiate the surface of the object to be measured;

[0013] The arc-shaped light source includes a plurality of LED lights, and each area is respectively connected to the control unit;

[0014] The control unit is connected to an image acquisition unit and a stepping motor;

[0015] The image acquisition unit is located on the side of the arc-shaped light source away from the object to be measured; the image acquisition unit acquires images each time the rotary table rotates into place;

[0016] The control unit controls the stepping motor to rotate by a specified angle, the lighting and extinguishing of the LED lights in each area of the arc-shaped light source, and triggers the image acquisition unit to acquire images.

[0017] Optionally, by controlling the lighting and extinguishing of the LED lights in each area of the arc-shaped light source and the rotation angle of the crankshaft to be measured, complete and non-overlapping images are obtained to eliminate the influence of different reflection characteristics of arc surfaces with different curvatures on defect detection, and by controlling the precise positioning of the rotary table, all defects on high-reflection complex curved surfaces can be collected without repeated detection.

[0018] Optionally, each area controlled by the arc-shaped light source is defined as follows: 10 lamp beads in the same row are divided into the same area.

[0019] Optionally, the image acquisition unit includes a camera and a lens, the camera and the lens are connected, and the signal line of the camera is connected to the control unit.

[0020] Optionally, the camera is a black-and-white CMOS industrial camera.

[0021] A method for defect detection of a high-reflection machined surface based on the device according to any one of the above, the method comprising the following steps:

[0022] First, extract the bright area parts in the image and splice them into a single image; wherein, the range of the bright area parts is determined according to the above-mentioned defect detection device;

[0023] Secondly, obtain an initial defect area after processing by Opencv; wherein, the Opencv processing operations include Gaussian filtering, threshold segmentation, dilation operation and intensity normalization; wherein, the initial defect area is a closed area obtained after the Opencv operations.

[0024] Thirdly, expand the upper, lower, left and right boundaries of the minimum circumscribed rectangle of the initial defect area as the ROI area;

[0025] Finally, input the ROI area into a convolutional neural network to extract features and obtain the final result.

[0026] Optionally, the threshold segmentation includes: calculating a threshold and a binarization operation;

[0027] The algorithm for calculating the threshold is the OTSU algorithm, which divides all pixels into two categories: C1 greater than thresh and C2 less than thresh, and the formula is as follows:

[0028] p1m1 + p2m2 = m G (1)

[0029] p1 + p2 = 1 (2)

[0030] In the formula, p1 and p2 are the probabilities of C1 and C2 respectively, m1 and m2 are the means of C1 and C2 respectively, and m G is the mean of all pixels

[0031] According to the concept of variance, the between-class variance expression is obtained:

[0032] σ 2 = p1p2(m1 - m2) 2 (3)

[0033]

[0034]

[0035]

[0036] Traverse all gray levels, and the k that maximizes equation (3) is the OTSU threshold.

[0037] Optionally, the minimum bounding rectangle for expanding the initial defect area is specifically:

[0038] First, use the superpixel segmentation method to segment the original image;

[0039] Then, for each side of the bounding rectangle, select the superpixel area that is located on that side and has the largest overlapping area with the rectangle;

[0040] Finally, extend each side of the box to the boundary of the selected superpixel area.

[0041] Optionally, the superpixel segmentation algorithm is the SLIC algorithm;

[0042] The intensity normalization method is Min-max, and the formula is as follows:

[0043]

[0044] where input is the intensity of the input pixel, max is the maximum pixel intensity in the local area, and min is the minimum pixel intensity in the local area.

[0045] Optionally, in the convolutional neural network, the steps for refining the defect area include:

[0046] First, obtain the publicly available steel defect dataset to pre-train the model;

[0047] Secondly, collect the crankshaft surface images containing defects and manually annotate the data;

[0048] Thirdly, input the collected images to fine-tune the network model;

[0049] Finally, input the images into the trained neural network for forward inference to obtain the segmentation results.

[0050] In the convolutional neural network, forward inference includes the following steps:

[0051] First, input the ROI region into the backbone network to extract features, and obtain feature maps C1, C2, C3, and C4 respectively; among them, the feature maps C1, C2, C3, and C4 are the outputs after one, two, three, and four downsamplings respectively;

[0052] Then, input the feature maps C1, C2, C3, and C4 into the multi-scale feature fusion network and output the feature map P1;

[0053] Finally, upsample the feature map P1 to the original image size and then input it into the Softmax layer to predict each pixel.

[0054] The backbone network consists of a depthwise separable convolutional layer, a common convolutional layer, a normalization layer, an activation layer, and a Softmax layer.

[0055] The multi-scale feature fusion network includes upsampling, weight fusion, skip connection, and convolution;

[0056] The loss function is:

[0057]

[0058] where M is the number of samples in a single batch, N is the total number of pixels in each image, K is the number of categories, w k is the weight, is the probability of the k-th category for the j-th pixel, is the label value of the j-th pixel.

[0059] The beneficial effects of the present invention are as follows:

[0060] (1) The present invention proposes an illumination method of arc surface light source partitioned light control, which has a simple structure and can simulate the illumination of light sources at different angles, maximizing the retention of image features and facilitating subsequent image processing.

[0061] (2) The present invention uses traditional image processing methods to initially locate the defective area, reduce the amount of data input into the neural network image, and improve the inference speed.

[0062] (3) The present invention adopts depthwise separable convolution and skip connections to obtain a lightweight backbone network, effectively reducing the number of parameters.

[0063] (4) The present invention uses skip connections, upsampling, and weight fusion to implement a feature fusion network, fusing high-semantic information on low-level feature maps to improve the recognition accuracy.

[0064] Other advantages, objectives, and features of the present invention will, to some extent, be described in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail preferably with reference to the accompanying drawings, where:

[0066] Figure 1 is a system and method flowchart for defect detection of a highly reflective machining surface provided by an example of the present invention;

[0067] Figure 2 is a schematic diagram of a three-dimensional model of a crankshaft provided by an example of the present invention;

[0068] Figure 3 is a schematic diagram of a rotating table in a defect detection device provided by an example of the present invention;

[0069] Figure 4 is a schematic diagram of an arc surface light source provided by an example of the present invention;

[0070] Figure 5 is a schematic diagram of the control unit in a defect detection device provided by an example of the present invention adjusting the LED lighting area;

[0071] Figure 6 is a schematic diagram of expanding an initial defect provided by an example of the present invention;

[0072] Figure 7 is a schematic diagram of a backbone network structure provided by an example of the present invention;

[0073] Figure 8 is a schematic diagram of a multi-scale feature fusion network structure provided by an example of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0074] The following specific examples illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0075] Among them, the attached drawings are only for illustrative purposes, showing only schematic diagrams rather than real diagrams, and should not be construed as a limitation to the present invention; in order to better illustrate the embodiments of the present invention, some components in the attached drawings will be omitted, enlarged or reduced, which do not represent the dimensions of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the attached drawings may be omitted.

[0076] In the attached drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the attached drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the attached drawings are only for illustrative purposes and should not be construed as a limitation to the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific situations.

[0077] Please refer to Figures 1 to 8 , which is a device and method for detecting defects on a highly reflective processing surface.

[0078] On one hand, an embodiment of the present invention provides a defect detection device, including a rotating table, an image acquisition unit, an arc-shaped light source, and a control unit; the rotating table is used to precisely control the irradiation position of the object to be measured, and the rotating table includes a stepping motor, a reducer, a coupling, a flange shaft, a transition plate, a tailstock, a motor base, a bottom plate, a support frame, and a quick clamping device; the quick clamping device is composed of a tailstock, a center, and a fixing pin; the fixing pin is connected to a locking plate and functions to position and transmit torque; the center cooperates with the tailstock to fix both ends of the object to be measured, meeting the requirement of quick clamping and being applicable to crankshafts of different lengths; the reducer adopts a worm and worm gear reduction and is connected to one end of the quick clamping device through a coupling; the arc-shaped light source is placed directly above the object to be measured and is used to irradiate the surface of the object to be measured; the arc-shaped light source includes a plurality of LED lights, and each area is respectively connected to the control unit; the control unit is connected to the image acquisition unit and the stepping motor; the image acquisition unit is located on the side of the arc-shaped light source away from the object to be measured; the image acquisition unit acquires images each time the rotating table rotates to the designated position; the control unit controls the stepping motor to rotate by a specified angle, controls the lighting and extinguishing of the LED lights in each area of the arc-shaped light source, and triggers the image acquisition unit to acquire images.

[0079] According to the defect detection device provided by the embodiment of the present invention, by controlling the lighting and extinguishing of the LED lights in each area of the arc-shaped light source and the rotation angle of the crankshaft to be measured, a complete and non-overlapping image can be obtained to eliminate the influence of different reflection characteristics of arc surfaces with different curvatures on defect detection, and by controlling the precise positioning of the rotating table, it is ensured that all defects on the highly reflective complex curved surface can be collected without repeated detection.

[0080] According to an embodiment of the present invention, each area of the arc-shaped light source includes: 10 lamp beads in the same row are divided into the same area;

[0081] According to an embodiment of the present invention, the image acquisition unit includes a camera and a lens, the camera is connected to the lens, and the signal line of the camera is connected to the control unit.

[0082] According to an embodiment of the present invention, the camera is a black and white CMOS industrial camera.

[0083] Another embodiment of the present invention provides a defect detection method, including the following steps: First, extract the bright area parts in the image and splice them into a single image; wherein, the range of the bright area parts is determined according to the above-mentioned defect detection device; Second, obtain the initial defect area after processing by Opencv; wherein, the Opencv processing operations include Gaussian filtering, threshold segmentation, dilation operation and intensity normalization; wherein, the initial defect area is the closed area obtained after the Opencv operation; Third, expand the upper, lower, left and right boundaries of the minimum bounding rectangle of the initial defect area as the ROI area; Finally, input the ROI area into the convolutional neural network to extract features and obtain the final result.

[0084] According to an embodiment of the present invention, the threshold segmentation includes: calculating a threshold and binaryzation operation.

[0085] According to an embodiment of the present invention, the algorithm for calculating the threshold is the OTSU algorithm, which divides all pixels into two categories: C1 greater than thresh and C2 less than thresh, and the following formula can be obtained:

[0086] p1m1 + p2m2 = m G (1)

[0087] p1 + p2 = 1 (2)

[0088] In the formula, p1 and p2 are the probabilities of C1 and C2 respectively, m1 and m2 are the means of C1 and C2 respectively, and m G is the mean of all pixels

[0089] According to the concept of variance, the between-class variance expression can be obtained:

[0090] σ 2 = p1p2(m1 - m2) 2 (3)

[0091]

[0092]

[0093]

[0094] Traverse all gray levels, and the k that makes formula (3) the largest is the OTSU threshold.

[0095] According to an embodiment of the present invention, the method for expanding the minimum bounding rectangle of the initial defect area includes the following steps: First, segment the original image using the superpixel segmentation method; Then, for each side of the bounding rectangle, select the superpixel area that is located on that side and has the largest overlapping area with the rectangle; Finally, extend each side of the box to the boundary of the selected superpixel area.

[0096] According to an embodiment of the present invention, the superpixel segmentation algorithm is the SLIC algorithm, which was proposed in the paper "SLIC superpixels compared to state-of-the-art superpixel methods".

[0097] According to an embodiment of the present invention, the intensity normalization method is Min-max, and the formula is as follows:

[0098]

[0099] where input is the intensity of the input pixel, max is the maximum pixel intensity in the local area, and min is the minimum pixel intensity in the local area.

[0100] According to an embodiment of the present invention, the steps for refining the defective area by the convolutional neural network include: first, obtaining a publicly available steel defect dataset to pre-train the model; second, collecting crankshaft surface images containing defects and manually annotating the data; third, inputting the collected images to fine-tune the network model; and finally, inputting the images into the trained neural network for forward inference to obtain the segmentation result.

[0101] According to an embodiment of the present invention, the steps for forward inference of the convolutional neural network are as follows: first, input the ROI area into the backbone network to extract features, and obtain feature maps C1, C2, C3, and C4 respectively; among them, the feature maps C1, C2, C3, and C4 are the outputs after one, two, three, and four downsamplings respectively; then, input the feature maps C1, C2, C3, and C4 into the multi-scale feature fusion network and output the feature map P1; finally, upsample the feature map P1 to the original image size and input it into the Softmax layer to predict each pixel.

[0102] According to an embodiment of the present invention, the backbone network includes a depthwise separable convolutional layer, a common convolutional layer, a normalization layer, an activation layer, and a Softmax layer.

[0103] According to an embodiment of the present invention, the multi-scale feature fusion network includes upsampling, weight fusion, skip connection, and convolution.

[0104] According to an embodiment of the present invention, the loss function is:

[0105]

[0106] where M is the number of samples in a single batch, N is the total number of pixels in each image, K is the number of categories, w k is the weight, is the probability of the jth pixel belonging to the kth class, is the label value of the j-th pixel.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A device for detecting defects on a highly reflective machining surface, characterized in that: The device includes a rotating table, an image acquisition unit, an arc surface light source and a control unit; The rotating table is used to precisely control the irradiation position of the crankshaft to be measured; The rotating table includes a stepping motor, a reducer, a coupling, a flange shaft, a transition plate, a tailstock, a motor base, a bottom plate, a support frame and a quick clamping device; The quick clamping device consists of a tailstock, a center and a fixing pin; The fixing pin is connected to the locking plate and plays a role in positioning and transmitting torque; The center cooperates with the tailstock to fix both ends of the crankshaft to be measured, meeting the requirements of quick clamping and being applicable to crankshafts of different lengths; The reducer adopts worm and worm gear reduction and is connected to one end of the quick clamping device through a coupling; The arc surface light source is placed directly above the crankshaft to be measured, and the arc surface light source is used to irradiate the surface of the crankshaft to be measured; The arc surface light source includes a plurality of LED lights, and each area is respectively connected to the control unit; The control unit is connected to the image acquisition unit and the stepping motor; The image acquisition unit is located on the side of the arc surface light source away from the crankshaft to be measured; the image acquisition unit acquires images each time the rotating table rotates in place; The control unit controls the stepping motor to rotate by a specified angle, the on / off of the LED lights in each area of the arc surface light source, and triggers the image acquisition unit to acquire images; By controlling the on / off of the LED lights in each area of the arc surface light source and the rotation angle of the crankshaft to be measured, complete and non-overlapping images are obtained to eliminate the influence of different reflection characteristics of arc surfaces with different curvatures on defect detection, and through precise positioning control of the rotating table, all defects on the highly reflective complex surface can be collected without repeated detection.

2. The device for defect detection of a highly reflective machining surface according to claim 1, characterized in that: The control of each area of the arc surface light source is as follows: 10 lamp beads in the same row are divided into the same area.

3. The device for detecting defects on a highly reflective machining surface according to claim 1, wherein: The image acquisition unit includes a camera and a lens, the camera and the lens are connected, and the signal line of the camera is connected to the control unit.

4. A device for detecting defects on a highly reflective machining surface according to claim 3, characterized in that: The camera is a black and white CMOS industrial camera.

5. A method for detecting defects on a highly reflective machining surface based on the device according to any one of claims 1 to 4, characterized in that: The method includes the following steps: First, extract the bright area part in the image and splice it into a single image; among them, the range of the bright area part is determined according to the above defect detection device; Secondly, obtain the initial defect area after processing by Opencv; among them, the Opencv processing operations include Gaussian filtering, threshold segmentation, dilation operation and intensity normalization; among them, the initial defect area is the closed area obtained after the Opencv operation; Thirdly, expand the upper, lower, left and right boundaries of the minimum circumscribed rectangle of the initial defect area as the ROI area; Finally, input the ROI area into the convolutional neural network to extract features and obtain the final result.

6. The method for detecting defects on a highly reflective machining surface according to claim 5, characterized in that: The threshold segmentation includes: calculating the threshold and binaryzation operation; The algorithm for calculating the threshold is the OTSU algorithm, which divides all pixels into two categories: C1 greater than thresh and C2 less than thresh, and the formula is as follows: p1m1 + p2m2 = m G (1) p1 + p2 = 1 (2) where p1 and p2 are the probabilities of C1 and C2 respectively, m1 and m2 are the means of C1 and C2 respectively, and m G is the mean of all pixels According to the concept of variance, the between-class variance expression is obtained: σ 2 = p1p2(m1 - m2) 2 (3) Traverse all gray levels, and the k that makes formula (3) the largest is the OTSU threshold.

7. The method for detecting defects on a highly reflective machining surface according to claim 6, characterized in that: The expansion of the minimum circumscribed rectangle of the initial defect area is specifically as follows: First, use the superpixel segmentation method to segment the original image; Then, for each side of the circumscribed rectangle, select the superpixel region that lies on that side and has the largest overlapping area with the rectangle; Finally, extend each side of the bounding box to the boundary of the selected superpixel region.

8. The method for defect detection of a highly reflective machined surface according to claim 7, wherein: The superpixel segmentation method is the SLIC algorithm; The intensity normalization method is Min-max, and the formula is as follows: where input is the intensity of the input pixel, max is the maximum pixel intensity in the local region, and min is the minimum pixel intensity in the local region.

9. The method for defect detection of a highly reflective machining surface according to claim 8, characterized in that: In the convolutional neural network, the steps for refining the defective region are as follows: First, obtain a publicly available steel defect dataset to pre-train the model; Second, collect crankshaft surface images containing defects and manually annotate the data; Third, input the collected images to fine-tune the network model; Finally, input the images into the trained neural network for forward inference to obtain the segmentation result; In the convolutional neural network, forward inference includes the following steps: First, input the ROI region into the backbone network to extract features, and obtain feature maps C1, C2, C3, and C4 respectively; among them, the feature maps C1, C2, C3, and C4 are the outputs after one, two, three, and four downsamplings respectively; Then, input the feature maps C1, C2, C3, and C4 into the multi-scale feature fusion network and output the feature map P1; Finally, upsample the feature map P1 to the original image size and then input it into the Softmax layer to predict each pixel; The backbone network consists of a depthwise separable convolutional layer, a common convolutional layer, a normalization layer, an activation layer, and a Softmax layer; The multi-scale feature fusion network includes upsampling, weight fusion, skip connection, and convolution; The loss function is: where M is the number of samples in a single batch, N is the total number of pixels in each image, K is the number of classes, w k is the weight, is the probability of the j-th pixel belonging to the k-th class, is the label value of the j-th pixel.

Images