Self-adaptive workpiece defect detection method based on machine vision, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of machine vision defect detection, and particularly discloses a self-adaptive workpiece defect detection method based on machine vision, electronic equipment and a storage medium, and the method comprises the following steps: S1, shooting an original picture of a workpiece, and processing the original picture to obtain a cut picture; s2, acquiring an end face template picture in the shot picture, and acquiring an end face area of the workpiece; s3, constructing a defect detection algorithm for detecting surface defects of the workpiece; s4, constructing a self-adaptive algorithm selector, and detecting surface defects of the workpiece by adopting a proper defect detection algorithm; according to the method, the images acquired by the image acquisition equipment are subjected to operations such as preprocessing and adaptive algorithm selection, so that whether the surface of the workpiece has defects can be detected, a proper scratch detection scheme can be selected according to detection results of workpieces of different batches, the method can adapt to workpieces of different types and batches, detection of defects of different types is met, and the detection efficiency is improved. And the defect detection accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision defect detection, and in particular relates to an adaptive workpiece defect detection method based on machine vision, an electronic device and a storage medium. Background Art

[0002] During the processing and manufacturing of existing workpieces, various defects will occur, such as scratches, unpolished parts, and incomplete appearance. Defective workpieces will seriously affect the performance and service life of the product. Therefore, workpiece defect detection is an indispensable part of manufacturing and one of the important means to improve product quality under existing conditions.

[0003] Most of the existing workpieces are still inspected for defects using manual visual inspection. However, manual inspection has problems such as poor consistency in judgment standards among different workers, fatigue caused by long hours of work, and reduced inspection quality. In addition, data statistical analysis is difficult during manual inspection, which is not conducive to further resource allocation, production process adjustment, and other work based on production data, and is not conducive to further automation and intelligence of industrial production.

[0004] With the development of machine vision technology, the use of machine vision technology to achieve automated inspection of product workpieces has gradually become the mainstream method of defect detection; machine vision has the advantages of good real-time performance, high precision, and non-contact. It can effectively improve the efficiency and accuracy of defect detection without affecting production, reduce labor costs, and there is no problem of inconsistent judgment standards under a unified algorithm, which is conducive to the automation and intelligence of product production.

[0005] Traditional defect detection methods based on machine vision require algorithm designers to design algorithms to extract feature information from images, and then determine whether there are defects in the image based on the feature information, and mark the location of the defects as much as possible. Existing defect detection methods based on machine vision mostly use methods based on deep learning. In the process of actual workpiece defect detection, deep learning methods also have certain limitations. All deep learning methods rely on the annotation of a large amount of data without exception. However, in the actual factory production process, the number of defective parts is very small and far less than the number of normal workpieces. It is very difficult to obtain a large amount of labeled data. In this case, traditional methods are still meaningful methods, and if the manually designed feature extraction method is appropriate, traditional methods can often achieve faster detection speeds than deep learning methods.

[0006] How to deal with multiple batches of workpieces, many types of defects and few defect samples is a difficult problem that needs to be solved urgently in the process of intelligent defect detection using machine vision. Based on this, the present invention proposes an adaptive workpiece defect detection method based on machine vision. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides a machine vision-based adaptive workpiece defect detection method, electronic device and storage medium that meet the needs of different batches of workpiece defect detection.

[0008] Based on the above purpose, the present invention is achieved through the following technical solutions:

[0009] A first aspect of the present invention provides an adaptive workpiece defect detection method based on machine vision, comprising the following steps:

[0010] S1. Take an original picture of the workpiece, process the original picture, and obtain a cropped picture.

[0011] S2. Obtain an end face template image in the captured image to obtain the end face area of ​​the workpiece.

[0012] S3. Construct a defect detection algorithm to detect surface defects of workpieces.

[0013] S4. Build an adaptive algorithm selector and use a suitable defect detection algorithm to detect surface defects of the workpiece.

[0014] According to the above-mentioned adaptive workpiece defect detection method based on machine vision, preferably, in step S1, in order to ensure that scratches with very fine width can also be detected, it is necessary to use a high-resolution camera to collect the original picture of the workpiece, the resolution of the camera is 4024×3036, and the OSTU method is used to process the original picture to obtain a cropped picture containing the workpiece, which is used to eliminate the large background area and interference content in the original picture; specifically: when collecting the original picture, the workpiece in the original picture does not fill the entire picture, and there are background areas in the original picture that are not related to the workpiece, and there are stains, light and shadow in the background area, which will constitute interference content for defect detection; therefore, if defect detection is performed directly on the original picture, not only will the unnecessary amount of calculation be greatly increased, but defects may also be detected in the background area.

[0015] Since the grayscale of the workpiece is relatively low and the grayscale of the background is relatively high in the image, a simpler way is to directly use threshold binarization to obtain the binary contour of the workpiece image.

[0016] However, there are differences in the grayscale values ​​of different workpieces, and on the same workpiece, due to the uneven illumination of low-angle ring light, using a fixed threshold will lead to the risk of incomplete capture of the workpiece; the OSTU method can adaptively select the grayscale threshold to distinguish between foreground and background; after improvement, the OSTU method can further reduce the amount of calculation; this cropping method has no requirements on the shape, lighting conditions, material and reflectivity of the workpiece surface, and can be applied to the cropping of workpieces of various shapes, and shows a certain robustness to background interference.

[0017] According to the above-mentioned adaptive workpiece defect detection method based on machine vision, preferably, in step S2, it is most convenient to obtain the end face area by means of edge extraction; however, due to the presence of shadows in the captured workpiece image, the edges are not very clear, and the effect is often not good if the edge extraction method is used; considering that the shape of the workpiece is known before production, and there are standard shape features such as circles and straight lines, a standard workpiece end face binary image can be first obtained as a template, and matched to the cropped image through rigid transformation to obtain the end face template image in the cropped image, specifically: in order to achieve matching, it is necessary to obtain the relative size relationship and position relationship between the template image and the workpiece in the cropped image. Since there are obvious circle and straight line features in most cropped images, the circles and straight lines in the cropped images are identified, and the positions of the circles and straight lines can be used to describe the position, placement direction and size of the workpiece in the image, and the expression is a 5-dimensional vector matchVec, which is expressed as:

[0018] matchVec = [cx cy lx ly f];

[0019] Among them, cx represents the x-coordinate of the center of the circle in the figure; cy represents the y-coordinate of the center of the circle in the figure; lx is the x-coordinate of the foot of the perpendicular line made by the straight line farthest from the center of the circle through the center of the circle, and ly is the y-coordinate of the foot of the perpendicular line; f represents a binary quantity, or a Boolean quantity, indicating whether the workpiece is on the front or back.

[0020] For simple workpieces, such as flanges, since the workpiece is composed entirely of circles, it is only necessary to use a circle detection algorithm to obtain the center and radius of the circle in the flange to obtain the end face area of ​​the flange.

[0021] According to the above-mentioned adaptive workpiece defect detection method based on machine vision, preferably, in step S3, the defect detection algorithm includes a stamping workpiece defect detection algorithm based on template comparison, a stamping part scratch detection algorithm based on gradient, and a defect detection algorithm based on deep learning.

[0022] According to the above-mentioned adaptive workpiece defect detection method based on machine vision, preferably, in step S3, the steps of constructing a stamping workpiece defect detection algorithm based on template comparison are: by transforming the spatial relationship between the template image and the cropped image to be detected, alignment is achieved, and then the features are compared to obtain the detection result of the surface defects of the workpiece; in step S2, the end face area of ​​the workpiece is obtained by calculating the matchVec vector; due to the existence of low-grayscale shadows and environmental interference, neither the edge detection-based method nor the threshold-based method can accurately obtain the end face of the workpiece; however, the shape of the workpiece is known after all; by deliberately placing the workpiece on a clean background, using a shadowless uniform light source to illuminate the workpiece (a clean background is difficult to achieve at the detection site, and uniform light cannot capture defects such as scratches), and photographing the workpiece, an ideal template image can be obtained; the end face of the workpiece on the template image can be mapped to the cropped image to be detected as long as it is rotated, translated, and scaled; further based on contour feature comparison, detect whether the workpiece is defective.

[0023] According to the above-mentioned adaptive workpiece defect detection method based on machine vision, preferably, in step S3, scratches are a defect that often appears on stamping parts, usually in linear shape, often appearing as bright lines under low-angle ring light, but often difficult to see from the picture under uniform light; scratches almost never appear near the edge of the workpiece in a direction parallel to the edge of the workpiece, and the grayscale near the edge of the workpiece often changes dramatically, and often appears as bright lines due to chamfer reflection, which can easily cause false detection; in step S2, a method of using a template image for rigid transformation to cover the end face of the cropped image to be detected has been given; when performing scratch detection, in order to avoid these easily falsely detected areas at the edge of the workpiece, it is necessary to perform an erosion operation on the end face mask obtained by rigid transformation of the template image.

[0024] The steps of constructing a gradient-based scratch detection algorithm for stamping parts are as follows: first, gradient-based suspicious pixel screening is performed in the stamping part scratch detection, and then the scratches on the stamping parts are determined and located. The suspicious points in the scratch area are roughly distributed near the straight line, which is an important basis for screening scratch pixel points and irrelevant interference points; although the suspicious points in the scratch area are also relatively fragmented, they are generally of a certain length and in the same direction; in this case, compared with interference points, scratch points are more easily identified as a straight line segment by the PPHT (cumulative probability Hough transform) algorithm; as long as a larger maximum straight line gap is set and the cumulative probability Hough transform algorithm is used for the first suspicious point screening, suspicious points densely arranged on the straight line can be screened out from the suspicious points, thereby excluding irrelevant interference points that are irregularly scattered on the entire end surface; the probability cumulative Hough transform algorithm is used again, while reducing the maximum straight line gap and slightly increasing the minimum straight line length, the irrelevant interference points left in the first screening can be excluded, leaving the real scratches, and obtaining the detection results of the surface defects of the workpiece.

[0025] According to the above-mentioned adaptive workpiece defect detection method based on machine vision, preferably, in step S3, in view of the diversity of workpiece surface defects and the problem that it is difficult to completely detect them using artificial features and prior knowledge, a defect detection algorithm based on deep learning is constructed. Since scratches on stamping parts are often long and thin lines, if a box is used to mark the scratches on the stamping parts, most of the boxes contain defect-free workpiece end face areas, and there may even be a considerable area of ​​background areas that are completely unrelated to the workpiece. Such marking is bound to mislead the detection; therefore, target detection is not the most suitable network for scratch defect detection, and semantic segmentation can minimize the interference of parts other than defects in the mark to the judgment; the steps of constructing a defect detection algorithm based on deep learning are: based on the YOLOv8 network with segmentation function, the workpiece end face area is further labeled and enhanced to obtain a data set, and the data set is put into the YOLOv8 segmentation model for 200 rounds of training. During training, the input image is reduced while keeping the aspect ratio unchanged, and the values ​​in the width and height are adjusted so that the larger value in the width and height is adjusted to 640, and automatic mixed precision calculation is disabled to improve the calculation accuracy, and the detection result of the workpiece surface defects is obtained.

[0026] According to the above-mentioned adaptive workpiece defect detection method based on machine vision, preferably, in step S4, the defect detection algorithm based on template matching, gradient and the method based on the YOLO model have their own advantages and disadvantages; the gradient-based method has a faster processing speed and has a higher detection rate for all scratch defects with obvious gradient changes and in a straight line, but the detection accuracy in the data set is lower than that of the YOLO model; the YOLO model has a relatively high accuracy in the data set, but the processing is relatively slow, and the detection effect for scratches that are significantly different from the data set pattern is poor.

[0027] In order to improve the detection accuracy, the algorithm is automatically selected according to the characteristics of the image, and an adaptive algorithm selector is constructed; according to the fact that the frequency of defects in a period of time in actual production is certain; when the defect frequency counted during the algorithm detection in the defect detection algorithm is closer to the factory's experience frequency, it means that the algorithm is more suitable for the needs of factory on-site production under the current circumstances, and an adaptive algorithm selector is constructed accordingly; the purpose of the adaptive algorithm selector is to find the algorithm that is more suitable for the needs of the factory; the workflow of the adaptive algorithm selector is as follows: before working, you need to enter the expected defect frequency v e ; When the adaptive algorithm selector is working, all defect detection algorithms are run; for any algorithm i∈I in the existing algorithm pool I, after a period of operation, the defect detection rate v during this period is calculated i , then |v i -v e| is the deviation between the defect detection rate of algorithm i and the expected frequency, and the algorithm i finally selected * for:

[0028]

[0029] The algorithm that adapts to the factory needs continues to run, all other algorithms stop running, the adaptive algorithm selector completes its work, and the workpiece defects are detected.

[0030] For the defect detection algorithm designed for stamping parts, due to the complex shape of stamping parts and the serious shadow interference in the pictures after shooting, it is necessary to first use circle detection and line detection algorithms to obtain the position information of the feature points of the workpiece, and then use the position information to calculate the rigid transformation matrix, map the existing end face template image to the cropped image of the image to be detected, and obtain the end face area of ​​the cropped image; the mapped template can be compared with the source image to realize the defect detection of the workpiece; the mapped template can be used as a mask to further perform gradient-based scratch detection; in order to try to further improve the accuracy of scratch detection, a scratch detection algorithm based on the YOLOv8 segmentation model is used; based on the constructed algorithm library, an adaptive algorithm selector is designed; the adaptive algorithm selector can automatically select the scratch detection scheme according to the detection results.

[0031] In practical applications, an adaptive algorithm selector is used and deployed in the defect detection software system to form an adaptive workpiece defect detection method based on machine vision, which is suitable for different types of workpieces and different defect detections, thereby improving the accuracy of defect detection.

[0032] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements any step in the adaptive workpiece defect detection method based on machine vision as described in the first aspect.

[0033] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer processor, the computer program implements any step of the adaptive workpiece defect detection method based on machine vision as described in the first aspect.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The present invention can detect whether there are defects on the surface of the workpiece by performing preprocessing and adaptive selection of algorithms on the images obtained by the image acquisition device, can select a suitable scratch detection scheme according to the detection results of different batches of workpieces, can adapt to different types and batches of workpieces, meet different types of defect detection, improve the accuracy of defect detection, and solve the problem of defect detection of different batches and types of workpieces.

[0036] 2. Design defect detection algorithm for stamping parts. Due to the complex shape of stamping parts and serious shadow interference in the pictures after shooting, we must first use circle detection and line detection algorithms to obtain the position information of the workpiece feature points, and then use the position information to calculate the rigid transformation matrix, map the existing end face template image to the cropped image of the image to be detected, and obtain the end face area of ​​the cropped image; the mapped template can be compared with the source image to realize the detection of workpiece defects; the mapped template can be used as a mask to further perform gradient-based scratch detection; in order to try to further improve the accuracy of scratch detection, a scratch detection algorithm based on the YOLOv8 segmentation model is used; based on the constructed algorithm library, an adaptive algorithm selector is designed; the adaptive algorithm selector can automatically select the scratch detection scheme according to the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the original picture of the workpiece;

[0038] Figure 2 It is a schematic diagram of a cropped picture of a workpiece;

[0039] Figure 3 It is a schematic diagram of defect detection results of stamping workpiece defect detection algorithm based on template comparison;

[0040] Figure 4 It is a schematic diagram of defect detection results of a scratch detection algorithm for stamping parts based on gradient;

[0041] Figure 5 This is a schematic diagram of defect detection results of a defect detection algorithm based on deep learning;

[0042] Figure 6 It is a schematic diagram of the workflow of the adaptive algorithm selector. DETAILED DESCRIPTION

[0043] The present invention is further described in detail below through specific examples, but the scope of the present invention is not limited thereto.

[0044] Example 1

[0045] An adaptive workpiece defect detection method based on machine vision comprises the following steps;

[0046] S1. Take an original picture of the workpiece, process the original picture, and obtain a cropped picture.

[0047] In order to ensure that even very fine scratches can be detected, a high-resolution camera is required to capture the original image of the workpiece. The camera resolution is 4024×3036. The OSTU method is used to process the original image to obtain a cropped image containing the workpiece, which is used to remove large background areas and interference content in the original image. Specifically, when the original image is captured, the workpiece does not fill the entire image. There are background areas in the original image that are not related to the workpiece, and there are stains, light and shadow in the background area that may interfere with defect detection. Therefore, if defect detection is performed directly on the original image, not only will the unnecessary amount of calculation be greatly increased, but defects may also be detected in the background area.

[0048] Since in the picture, the grayscale of the workpiece is relatively low, while the grayscale of the background is high, such as Figure 1 As shown, a simpler way is to directly use threshold binarization to obtain the binary contour of the workpiece image.

[0049] However, the grayscale values ​​of different workpieces are different, and on the same workpiece, due to the uneven illumination of low-angle ring light, using a fixed threshold will cause the risk of incomplete workpiece capture. The OSTU method can adaptively select the grayscale threshold to distinguish the foreground and background. After improvement, the OSTU method can further reduce the amount of calculation. The final cropping effect is as follows: Figure 2 As shown; the large background area in the picture is removed, leaving only the part with the workpiece; this cropping method has no requirements on the shape of the workpiece, lighting conditions, material and reflectivity of the workpiece surface, can be applied to the cutting of workpieces of various shapes, and shows a certain robustness to background interference.

[0050] S2. Obtain an end face template image in the captured image to obtain the end face area of ​​the workpiece.

[0051] It is most convenient to use edge extraction to obtain the end face area; however, due to the presence of shadows in the workpiece image, the edges are not very clear, and the effect of using edge extraction methods is often not good; considering that the shape of the workpiece is known before production, and there are standard shape features such as circles and straight lines, a standard workpiece end face binary image can be obtained as a template, and matched to the cropped image through rigid transformation to obtain the end face template image in the cropped image. Specifically: In order to achieve matching, it is necessary to obtain the relative size relationship and position relationship between the template image and the workpiece in the cropped image. Since there are obvious circle and straight line features in most cropped images, such as Figure 2 As shown, therefore, by identifying the circles and straight lines in the cropped image, the positions of the circles and straight lines can be used to describe the position, placement direction and size of the workpiece in the image. The expression is a 5-dimensional vector matchVec, which is expressed as:

[0052] matchVec = [cx cy lx ly f];

[0053] Among them, cx represents the x-coordinate of the center of the circle in the figure; cy represents the y-coordinate of the center of the circle in the figure; lx is the x-coordinate of the foot of the perpendicular line made by the straight line farthest from the center of the circle through the center of the circle, and ly is the y-coordinate of the foot of the perpendicular line; f represents a binary quantity, or a Boolean quantity, indicating whether the workpiece is on the front or back.

[0054] For simple workpieces, such as flanges, since the workpiece is composed entirely of circles, it is only necessary to use a circle detection algorithm to obtain the center and radius of the circle in the flange to obtain the end face area of ​​the flange.

[0055] S3. Construct a defect detection algorithm to detect surface defects of workpieces.

[0056] The defect detection algorithms include a stamping workpiece defect detection algorithm based on template comparison, a stamping part scratch detection algorithm based on gradient, and a defect detection algorithm based on deep learning.

[0057] The steps of constructing a stamping workpiece defect detection algorithm based on template comparison are as follows: by transforming the spatial relationship between the template image and the cropped image to be detected, alignment is achieved, and then the features are compared to obtain the detection results of the workpiece surface defects; in step S2, the end face area of ​​the workpiece is obtained by calculating the matchVec vector; due to the existence of low-grayscale shadows and environmental interference, neither the edge detection method nor the threshold-based method can accurately obtain the end face of the workpiece; however, the shape of the workpiece is known after all; by deliberately placing the workpiece on a clean background, using a shadowless uniform light source to illuminate the workpiece (a clean background is difficult to achieve at the detection site, and uniform light cannot capture defects such as scratches), and photographing the workpiece, an ideal template image can be obtained; the end face of the workpiece on the template image can be mapped to the cropped image to be detected as long as it is rotated, translated, and scaled; further based on contour feature comparison, detect whether the workpiece has defects, such as Figure 3 shown.

[0058] Scratches are a defect that often appears on stamping parts. They are usually linear and often appear as bright lines under low-angle ring light, but are often difficult to see from the picture under uniform light. Scratches rarely appear near the edge of the workpiece in a direction parallel to the edge of the workpiece, while the grayscale near the edge of the workpiece often changes dramatically, and often appear as bright lines due to chamfer reflections, which can easily cause false detection. In step S2, a method for using a template image to perform a rigid transformation to cover the end face of the cropped image to be detected has been given. When performing scratch detection, in order to avoid these easily falsely detected areas on the edge of the workpiece, it is necessary to perform an erosion operation on the end face mask obtained by rigid transformation of the template image.

[0059] The steps of constructing a gradient-based scratch detection algorithm for stamping parts are as follows: first, gradient-based suspicious pixel screening is performed in the stamping part scratch detection, and then the scratches on the stamping parts are determined and located. The suspicious points in the scratch area are roughly distributed near the straight line, which is an important basis for screening scratch pixel points and irrelevant interference points; although the suspicious points in the scratch area are also relatively fragmented, they are generally of a certain length and in the same direction; in this case, compared with interference points, scratch points are more easily identified as a straight line segment by the PPHT (cumulative probability Hough transform) algorithm; as long as a larger maximum straight line gap is set and the cumulative probability Hough transform algorithm is used for the first suspicious point screening, suspicious points densely arranged on the straight line can be screened out from the suspicious points, thereby excluding irrelevant interference points that are irregularly scattered on the entire end surface; the probability cumulative Hough transform algorithm is used again, while reducing the maximum straight line gap and slightly increasing the minimum straight line length, the irrelevant interference points left in the first screening can be excluded, leaving the real scratches, and the detection results of the surface defects of the workpiece can be obtained; from Figure 4 As you can see, the scratches in the image are accurately detected and the locations of the scratches are marked.

[0060] In view of the diversity of surface defects of workpieces and the difficulty in completely detecting them using artificial features and prior knowledge, a defect detection algorithm based on deep learning is constructed. Since scratches on stamping parts are often long and thin lines, if a box is used to mark the scratches on the stamping parts, most of the boxes contain defect-free end face areas of the workpiece, and there may even be a considerable area of ​​background areas that are completely unrelated to the workpiece. Such marking is bound to mislead the detection; therefore, target detection is not the most suitable network for scratch defect detection, and semantic segmentation can minimize the interference of parts other than defects in the mark; the steps for constructing a defect detection algorithm based on deep learning are as follows: based on the YOLOv8 network with segmentation function, the end face area of ​​the workpiece is further labeled and enhanced to obtain a data set, and the data set is put into the YOLOv8 segmentation model for 200 rounds of training. During training, the input image is reduced while keeping the aspect ratio unchanged, and the values ​​in the width and height are adjusted so that the larger value in the width and height is adjusted to 640, and automatic mixed precision calculation is disabled to improve the calculation accuracy, and the detection results of surface defects of the workpiece are obtained; the detection results are as follows Figure 5 shown.

[0061] S4. Build an adaptive algorithm selector and use a suitable defect detection algorithm to detect surface defects of the workpiece.

[0062] Defect detection algorithms based on template matching and gradient and methods based on the YOLO model each have their own advantages and disadvantages; the gradient-based method has a faster processing speed and a higher detection rate for all scratch defects with obvious gradient changes and in a straight line, but its detection accuracy in the data set is lower than that of the YOLO model; the YOLO model has a relatively high accuracy in the data set, but the processing is relatively slow, and the detection effect is poor for scratches that are significantly different from the data set pattern.

[0063] In order to improve the detection accuracy, the algorithm is automatically selected according to the characteristics of the image, and an adaptive algorithm selector is constructed. The workflow of the adaptive algorithm selector is as follows: Figure 6 As shown in the figure, in actual production, the frequency of defects is certain within a period of time. When the defect frequency counted by the algorithm detection in the defect detection algorithm is closer to the factory's experience frequency, it means that the algorithm is more suitable for the needs of factory production under the current situation. Based on this, an adaptive algorithm selector is constructed. The purpose of the adaptive algorithm selector is to find the algorithm that is more suitable for the needs of the factory. The workflow of the adaptive algorithm selector is as follows: before working, you need to enter the expected defect frequency v e ; When the adaptive algorithm selector is working, all defect detection algorithms are run; for any algorithm i∈I in the existing algorithm pool I, after a period of operation, the defect detection rate v during this period is calculated i , then |v i -v e | is the deviation between the defect detection rate of algorithm i and the expected frequency, and the algorithm i finally selected * for:

[0064]

[0065] The algorithm that adapts to the factory needs continues to run, all other algorithms stop running, the adaptive algorithm selector completes its work, and the workpiece defects are detected.

[0066] Defect detection algorithms designed for stamping parts, such as Figure 6As shown in the figure, due to the complex shape of the stamping parts and the serious shadow interference in the pictures after shooting, it is necessary to first use the circle detection and line detection algorithms to obtain the position information of the workpiece feature points, and then use the position information to calculate the rigid transformation matrix, map the existing end face template image to the cropped image of the image to be detected, and obtain the end face area of ​​the cropped image; the mapped template can be compared with the source image to realize the detection of workpiece defects; the mapped template can be used as a mask to further perform gradient-based scratch detection; in order to try to further improve the accuracy of scratch detection, a scratch detection algorithm based on the YOLOv8 segmentation model is used; based on the constructed algorithm library, an adaptive algorithm selector is designed; the adaptive algorithm selector can automatically select the scratch detection scheme according to the detection results.

[0067] In practical applications, an adaptive algorithm selector is used and deployed in the defect detection software system to form an adaptive workpiece defect detection method based on machine vision, which is suitable for different types of workpieces and different defect detections, thereby improving the accuracy of defect detection.

[0068] Example 2

[0069] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, any step of an adaptive workpiece defect detection method based on machine vision as described in Example 1 is implemented.

[0070] Further, the process of the adaptive workpiece defect detection method based on machine vision described in Embodiment 1 can be implemented as a computer software program. For example, this embodiment includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable medium. When the computer program is executed by a processor, the above functions defined in the method of the present application are executed.

[0071] Example 3

[0072] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step in an adaptive workpiece defect detection method based on machine vision as described in Example 1.

[0073] The computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0074] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0075] In summary, the present invention effectively overcomes the deficiencies in the prior art and has a high industrial utilization value. The above embodiments serve to illustrate the substantive content of the present invention, but do not limit the protection scope of the present invention. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and protection scope of the technical solution of the present invention.

[0076] The above embodiments are specific implementation methods of the present invention, but the implementation methods of the present invention are not limited to the above embodiments. Any other combination, change, modification, substitution and simplification that does not exceed the design concept of the present invention shall fall within the protection scope of the present invention.

Claims

1. An adaptive workpiece defect detection method based on machine vision, characterized in that: The following steps are involved: S1. Take an original picture of the workpiece, process the original picture, and obtain a cropped picture; S2, obtaining an end face template image in the captured image, and obtaining the end face area of ​​the workpiece; S3, constructing a defect detection algorithm for detecting surface defects of workpieces; S4. Build an adaptive algorithm selector and use a suitable defect detection algorithm to detect surface defects of the workpiece.

2. The method for adaptive workpiece defect detection based on machine vision according to claim 1, characterized in that: In step S1, a high-resolution camera is used to capture an original image of the workpiece, and the OSTU method is used to process the original image to obtain a cropped image containing the workpiece, which is used to eliminate large background areas and interference content in the original image.

3. The method for adaptive workpiece defect detection based on machine vision according to claim 2, characterized in that: In step S2, a standard workpiece end face binary image is first obtained as a template, and matched to the cropped image through rigid transformation to obtain the end face template image in the cropped image. Specifically, the relative size relationship and position relationship between the template image and the workpiece in the cropped image are obtained. Since there are obvious circle and straight line features in the cropped image, the circles and straight lines in the cropped image are identified, and the positions of the circles and straight lines can be used to describe the position, placement direction and size of the workpiece in the image. The expression is a 5-dimensional vector matchVec, which is expressed as follows: matchVec = [cx cy lx ly f]; Among them, cx represents the x-coordinate of the center of the circle in the figure; cy represents the y-coordinate of the center of the circle in the figure; lx is the x-coordinate of the foot of the perpendicular line made by the straight line farthest from the center of the circle through the center of the circle, and ly is the y-coordinate of the foot of the perpendicular line; f represents a binary quantity, or a Boolean quantity, indicating whether the workpiece is on the front or back.

4. The method for adaptive workpiece defect detection based on machine vision according to claim 3, characterized in that: In step S3, the defect detection algorithms include a stamping workpiece defect detection algorithm based on template comparison, a stamping part scratch detection algorithm based on gradient, and a defect detection algorithm based on deep learning.

5. The method for adaptive workpiece defect detection based on machine vision according to claim 4, characterized in that: In step S3, the steps of constructing a stamping workpiece defect detection algorithm based on template comparison are: by transforming the spatial relationship between the template image and the cropped image to be detected, alignment is achieved, and then features are compared to obtain the detection result of the workpiece surface defects.

6. The method for adaptive workpiece defect detection based on machine vision according to claim 5, characterized in that: In step S3, the steps of constructing a gradient-based scratch detection algorithm for stamping parts are as follows: first, gradient-based suspicious pixel screening is performed in the stamping part scratch detection, and then the stamping part scratches are determined and located; a larger maximum straight line gap is set, and the cumulative probability Hough transform algorithm is used to perform the first suspicious point screening, and suspicious points densely arranged on the straight line are screened out from the suspicious points, and irrelevant interference points that are irregularly scattered on the entire end surface are excluded; The probability cumulative Hough transform algorithm is used again, while reducing the maximum straight line gap, increasing the minimum straight line length, eliminating the irrelevant interference points left in the first screening, leaving the real scratches, and obtaining the detection results of the surface defects of the workpiece.

7. The method for adaptive workpiece defect detection based on machine vision according to claim 6, characterized in that: In step S3, the steps of constructing a defect detection algorithm based on deep learning are as follows: based on the YOLOv8 network with segmentation function, the end face area of ​​the workpiece is further labeled and enhanced to obtain a data set, and the data set is put into the YOLOv8 segmentation model for training. During training, the input image is reduced while keeping the aspect ratio unchanged, the values ​​in the width and height are adjusted, and the automatic mixed precision calculation is disabled to improve the calculation accuracy, so as to obtain the detection results of the surface defects of the workpiece.

8. The method for adaptive workpiece defect detection based on machine vision according to claim 7, characterized in that: In step S4, according to the fact that the frequency of defects in actual production is certain within a period of time, when the defect frequency counted in the defect detection algorithm is closer to the factory's experience frequency, it means that the algorithm is more suitable for the needs of factory on-site production under the current situation, and an adaptive algorithm selector is constructed accordingly; the workflow of the adaptive algorithm selector is as follows: before working, it is necessary to input the expected defect frequency v e ; When the adaptive algorithm selector is working, all defect detection algorithms are run; for any algorithm i∈I in the existing algorithm pool I, after a period of operation, the defect detection rate v during this period is calculated i , then |v i -v e | is the deviation between the defect detection rate of algorithm i and the expected frequency, and the algorithm i finally selected * for: The algorithm that adapts to the factory needs continues to run, all other algorithms stop running, the adaptive algorithm selector completes its work, and the workpiece defects are detected.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, any step of the adaptive workpiece defect detection method based on machine vision as described in any one of claims 1-8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer processor, it implements any step of the method for adaptive workpiece defect detection based on machine vision as described in any one of claims 1 to 8.

Images