Flange plate surface defect classification method and system based on machine vision

Abstract

The invention provides a flange plate surface defect classification method and system based on machine vision, and the method comprises the steps: 1, collecting an end face image of a flange plate; 2, determining an image contour of the flange plate end face based on a target algorithm; and step 3, extracting target Hu matrix features, performing training to generate a support vector machine, and performing defect classification on the image contour of the flange plate end face according to the support vector machine, thereby effectively improving the accuracy and robustness of edge detection, effectively providing more stable feature support through the target Hu matrix features, and improving the accuracy and robustness of edge detection through accurately training the support vector machine. And accurate defect classification of the flange plate labels can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a flange surface defect classification method based on machine vision and a system thereof. Background Art

[0002] At present, with the vigorous development of my country's economy and the continuous improvement of people's living standards, cars have gradually become an indispensable means of transportation in daily life. The rapid development of the automobile industry has put a huge test on the safety performance of automobiles. Flanges are one of the most important parts in automobiles. They are mainly used to connect the wheel hub and the axle to ensure that the wheel hub can be firmly installed on the axle. The quality of the flange directly determines the overall safety performance of the car.

[0003] However, in the production process of automotive flanges, the two main inspection methods used are human eye inspection and machine vision inspection. Among them, with the improvement of productivity and the urgent need for intelligent development, the efficiency of human eye inspection is extremely easy to reduce due to fatigue or other subjective factors, resulting in missed detection and false detection. Therefore, human eye inspection can no longer meet the high precision and high efficiency requirements of the current parts production inspection link; deep learning technology in machine vision has greatly improved the detection efficiency and accuracy, but deep learning has encountered two major difficulties in specific applications. On the one hand, its excellent performance is highly dependent on huge amounts of labeled data and high computing resource consumption. In reality, it is difficult to collect sufficient labeled data, which makes the effect of deep learning unsatisfactory when the data samples are limited; on the other hand, when computing resources are limited, the high demand for computing resources makes the cost of creating large-scale and diverse defect data sets extremely high. Especially when the frequency of defects is low, this process is not only time-consuming and labor-intensive, but may also cause unnecessary losses in production;

[0004] Therefore, in order to overcome the above technical problems, the present invention provides a flange surface defect classification method and system based on machine vision. Summary of the Invention

[0005] The present invention provides a flange surface defect classification method and system based on machine vision. In the denoising stage, a bilateral filtering algorithm is used to replace the traditional Gaussian filter to effectively suppress image noise while retaining edge information, thereby improving the algorithm's performance in the filtering process. In the gradient calculation phase, while the Sobel operator is still used as the core calculation method, it is expanded on this basis by adding detection templates in the 45° and 135° directions to enrich the extraction of edge information and enhance the comprehensiveness of edge detection. In the threshold determination process, the Otsu algorithm is introduced to automatically determine the adaptive dual threshold, effectively improving the accuracy and robustness of edge detection. After completing edge detection, the improved Hu rectangular feature is further extracted. While ensuring scale invariance, it also satisfies rotation and translation invariance, providing more stable feature support for subsequent defect classification. Finally, through the performance comparison of four kernel functions, the polynomial kernel function is selected to train the SVM to achieve accurate defect classification of the flange surface.

[0006] A flange surface defect classification method based on machine vision, comprising:

[0007] Step 1: Collect the flange end face image;

[0008] Step 2: Determine the image contour of the flange end face based on the target algorithm;

[0009] Step 3: Extract the target Hu matrix features and train them to generate a support vector machine. At the same time, classify the defects of the image contour of the flange end face based on the support vector machine.

[0010] Preferably, a flange surface defect classification method based on machine vision, in step 1, collecting an image of the flange end face, includes:

[0011] The flange end face image is captured by an industrial camera, and the obtained flange end face image is sent to a computer system through a USB interface. The flange end face image is transmitted to the computer end according to a driver, and the image is processed based on the computer end in an integrated development environment.

[0012] Preferably, a flange surface defect classification method based on machine vision, in step 2, the target algorithm is an improved Canny algorithm, wherein the improved Canny algorithm uses bilateral filtering instead of Gaussian filtering for denoising and adds gradient calculations in the 45° and 135° directions.

[0013] Preferably, a flange surface defect classification method based on machine vision, in step 2, determining the image contour of the flange end face based on a target algorithm, includes:

[0014] The flange end face image is pre-processed for denoising based on the target algorithm;

[0015] The flange end face image after denoising is positioned, the edge points of the flange end face image are determined, and the edge points are connected to determine the image contour of the flange end face.

[0016] Preferably, a flange surface defect classification method based on machine vision determines the edge point of the flange end face image, obtains the first gradient intensity given by the horizontal defense line of the edge point and the second gradient intensity in the edge direction respectively; compares the first gradient intensity with the second gradient intensity, and when the first gradient intensity is greater than the second extraction intensity, determines that the edge point of the flange end face image is selected as the gradient value in the horizontal direction; otherwise, selects the gradient value in the vertical direction.

[0017] Preferably, a flange surface defect classification method based on machine vision, performing denoising on the flange end face image includes:

[0018] S101: performing a grayscale operation on the flange end surface image to obtain a grayscale image, and setting a filter window according to a preset ratio;

[0019] S102: randomly selecting a central pixel in the grayscale image, and calculating the spatial distance weight between each pixel in the filter window and the central pixel;

[0020] S103: Calculate the gray value similarity weight between each pixel in the filter window and the central pixel;

[0021] S104: Multiplying the spatial distance weight corresponding to each pixel point and the central pixel point by the gray value similarity weight to obtain a comprehensive weight;

[0022] S105: Calculating the grayscale value of the filtered central pixel according to the comprehensive weight;

[0023] S106: Repeat steps S101 to S105 based on each pixel in the grayscale image to complete the image filtering operation.

[0024] Preferably, a flange surface defect classification method based on machine vision, in step 3, before extracting the target Hu matrix features, the order and scale factor of the normalized central moment in the original Hu matrix are adjusted to obtain the target Hu matrix.

[0025] Preferably, a flange surface defect classification method based on machine vision, step 3, includes: collecting a preset number of sample images, extracting Hu moment features corresponding to the target Hu matrix in the sample images, and training the Hu moment features according to the polynomial kernel function to obtain a support vector machine; and classifying defects in the image contour of the flange end face according to the support vector machine.

[0026] A flange surface defect classification system based on machine vision includes: an image acquisition module, a contour determination module and a defect classification module, wherein the image acquisition module acquires the flange end face image; the contour determination module determines the image contour of the flange end face based on a target algorithm; the defect classification module extracts the target Hu matrix features and trains it to generate a support vector machine, and at the same time, classifies the defects of the image contour of the flange end face according to the support vector machine.

[0027] Preferably, a flange surface defect classification system based on machine vision, an image acquisition module, captures the flange end face image according to the industrial camera, sends the obtained flange end face image to the computer system according to the USB interface, transmits the flange end face image to the computer end according to the driver, and processes the image based on the computer end in an integrated development environment.

[0028] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0029] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0031] Figure 1 This is a flow chart of a flange surface defect classification method based on machine vision in an embodiment of the present invention;

[0032] Figure 2 This is an algorithm flow chart of a flange surface defect classification method based on machine vision in an embodiment of the present invention;

[0033] Figure 3 This is a structural diagram of a flange surface defect classification system based on machine vision in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0035] In one embodiment, a flange surface defect classification method based on machine vision is provided. Figure 1 Shown, including:

[0036] Step 1: Collect the flange end face image;

[0037] The flange end face image is captured by an industrial camera, and the obtained flange end face image is sent to a computer system through a USB interface. The flange end face image is transmitted to the computer end according to a driver, and the image is processed based on the computer end using an integrated development environment, wherein the integrated development environment is the PyCharm integrated development environment; the processing includes: filtering and denoising, image segmentation and edge recognition.

[0038] Step 2: Determine the image contour of the flange end face based on the target algorithm;

[0039] The target algorithm is an improved Canny algorithm, wherein the improved Canny algorithm uses bilateral filtering instead of Gaussian filtering for denoising and adds gradient calculations in the directions of 45° and 135°, thereby improving the performance of image filtering and enhancing the extraction of edge information; based on the target algorithm, the flange end face image is preprocessed (i.e., bilateral filtering), and after filtering, the flange end face image is positioned using a gradient algorithm with added directions of 45° and 135° to determine the edge points of the flange end face image, and the edge points are connected to determine the image contour of the flange end face.

[0040] Step 3: Extract the target Hu matrix features and train them to generate a support vector machine. At the same time, classify the defects of the image contour of the flange end face based on the support vector machine.

[0041] A preset number of sample images are collected, and the Hu moment features corresponding to the target Hu matrix are extracted from these sample images. These Hu moment features are then trained using a polynomial kernel function to obtain a support vector machine. The support vector machine is then used to classify the flange end face image contours for defects. The preset number can be 300, and the sample images include three types: pinhole defects, scratch defects, and no defects. Polynomial kernel functions include linear kernel, Rbf kernel, Poly kernel, and Sigmoid kernel. The support vector machine is an SVM.

[0042] The present invention is a flange surface defect classification method based on machine vision, the specific algorithm flow chart is as follows Figure 2As shown in the figure, the following beneficial effects are achieved: in the denoising stage, the bilateral filtering algorithm is used to replace the traditional Gaussian filtering to effectively suppress image noise while preserving edge information, thereby improving the performance of the algorithm in the filtering process; in the gradient calculation stage, although the Sobel operator is still used as the core calculation method, it is expanded on this basis and detection templates in the 45° and 135° directions are added to enrich the extraction of edge information and enhance the comprehensiveness of edge detection; in the threshold determination process, the Otsu algorithm is introduced to automatically determine the adaptive dual threshold, effectively improving the accuracy and robustness of edge detection; after completing edge detection, the improved Hu rectangular feature is further extracted, which not only ensures scale invariance but also meets rotation and translation invariance, providing more stable feature support for subsequent defect classification. Finally, through the performance comparison of four kernel functions, the polynomial kernel function is selected to train the SVM to form the improved CannyH-SVM algorithm, which effectively improves the accuracy and robustness of edge detection. The target Hu matrix features effectively provide more stable feature support. By accurately training the support vector machine, it is conducive to accurate defect classification of flange labels.

[0043] In one embodiment, when collecting the flange end face image, due to the influence of the surrounding environment and the camera's own factors, the image often contains noise, which will affect the image segmentation and edge recognition. Therefore, in order to improve the quality of image processing, the flange end face image is subjected to bilateral denoising, specifically: S101: grayscale operation is performed on the flange end face image to obtain a grayscale image, and a filter window is set according to a preset ratio, where the preset ratio is, for example, 3×3; S102: a central pixel point is randomly selected in the grayscale image, and the spatial distance weight between each pixel point and the central pixel point in the filter window is calculated; S103: the grayscale value similarity weight between each pixel point and the central pixel point in the filter window is calculated; S104: the spatial distance weight corresponding to each pixel point and the central pixel point is multiplied by the grayscale value similarity weight to obtain a comprehensive weight; S105: the grayscale value of the filtered central pixel point is calculated according to the comprehensive weight; Among them, g(i,j) represents the grayscale value of the center pixel after filtering; i and j represent the coordinates of the center pixel; k and l represent the coordinates of the pixel points in the filtering window; ω(k,l,i,j) represents the comprehensive weight; f(k,l) represents the grayscale value of the pixel point (k,l) in the filtering window; S106: Repeat steps S101-S105 based on each pixel point in the grayscale image to complete the image filtering operation.

[0044] The bilateral filtering of the image is performed based on the above by adding 45° direction and 135° direction on the basis of the traditional Sobel horizontal and vertical directions.

[0045] When determining edge pixels, the direction of the gradient must also be considered. When determining edge points in a flange end-face image, the first gradient strength from the horizontal direction of the edge point and the second gradient strength in the edge direction are obtained. The first and second gradient strengths are compared. If the first gradient strength is greater than the second extracted strength, the horizontal gradient value is used to determine the edge point of the flange end-face image; otherwise, the vertical gradient value is used. For example, when the horizontal gradient strength (x-direction) exceeds the vertical gradient strength (y-direction), the judgment of whether the pixel is an edge will be based primarily on the horizontal gradient. Conversely, if the vertical gradient is more significant, the judgment will focus on the vertical gradient. By considering the gradient direction, the accuracy of edge recognition is improved.

[0046] In one embodiment, due to factors such as uneven lighting, the texture of the flange surface, or dust in the environment, the segmented image not only contains defect features but also is mixed with other interferences, which creates obstacles to the accurate extraction of defects. Therefore, after non-maximum suppression, there is often misjudged edge data in the image. In order to eliminate these effects, the Canny algorithm utilizes Otsu double threshold processing. The use of the Otsu segmentation algorithm can replace subjective experience to automatically determine the optimal double threshold. This method improves the convenience of threshold selection.

[0047] Specifically, the preset threshold t is obtained, and the flange end face image is divided into a background image and a foreground image;

[0048] Obtain the first probability p0 corresponding to the pixel being classified as the background image and the second probability p1 corresponding to the pixel being classified as the foreground image in the flange end face image, obtain the first grayscale mean m0 of the background image and the second grayscale mean m1 of the foreground image; calculate the global average grayscale value m of the flange end face image based on the first probability p0, the second probability p1, the first grayscale mean m0 and the second grayscale mean m1; construct the function σ 2 =p0(m0-m) 2 +p1(m1-m) 2 , when the function takes the maximum value, the maximum threshold and the minimum threshold are determined according to the preset threshold t.

[0049] Specific calculation formula: Assuming there is a threshold t that divides the image into two categories: background (represented by C0) and foreground (represented by C1), then the probability of a pixel being classified into C0 and C1 are p0 and p1 respectively, where the grayscale mean of C0 and C1 are m0 and m1 respectively, and the global average grayscale value of the image is m. The calculation is as follows: Where σ 2 Take the maximum value and select the optimal threshold t as the high threshold of the double threshold, and the low threshold is half of the high threshold.

[0050] In one embodiment, in order to provide more stable feature support for subsequent defect classification, the original Hu matrix is improved to obtain an improved target Hu matrix, including: first, calculating the (p+q)-order geometric moment and geometric central moment of the discrete image, secondly, calculating the central moment; and finally normalizing the central moment. When normalizing the central moment, the value of the normalized central moment is related to the order (p+q) and is affected by the scale factor σ. In order to maintain the invariance of the Hu moment to the scale factor in the discrete state, this paper improves the Hu moment based on h2, and constructs a set of new invariant moments as shown in the formula to complete the Hu matrix improvement. First, for the two-dimensional discrete image I(x,y), the (p+q)-order geometric moment and geometric central moment are calculated, and the formula is as follows: Where I(x,y) is the grayscale value of the image at point (x,y). Next, calculate the central moment μ pq Such as: Where, and is the coordinate of the image centroid, and the calculation formula is as follows Where M 00 is the zero-order moment, which represents the total area (total number of pixels) of the image. Then, the normalized central moment η is calculated pq , the calculation formula is as follows: Where μ 00 is the zero-order central moment of the image, equal to M 00 Finally, the normalized central moment is used to calculate the Hu moment. The calculation formulas for the 7 Hu moments are as follows The improvements include: when scaling a discrete image, the normalized central moment will also change, which is mainly expressed as follows, and the relationship between the 7 invariant moments under the participation of the scale factor σ is calculated as follows: As can be seen from the above formula, the value of the normalized central moment is related to the order (p+q) and is affected by the scale factor σ. In order to maintain the invariance of the Hu moment to the scale factor in the discrete state, this paper improves the Hu moment based on h2, and constructs a new set of invariant moments as shown in the formula: Where ρ is h, the value after eliminating the scaling factor σ. According to the above formula, in the discrete state, the above six eigenvalues can satisfy the invariance of scale, rotation and translation.

[0051] In step 2, in order to effectively ensure the integrity of the image contour of the flange end face and the comprehensiveness of the verification of the image contour of the flange end face, this embodiment also includes detecting the image contour of the flange end face, specifically: extracting the edge points of the current flange end face image contour, and randomly selecting the edge points as the coordinate center points to construct a plane rectangular coordinate system, and at the same time, mapping the edge points of the current flange end face image contour in the plane rectangular coordinate system to obtain the coordinate values of the edge points of the current flange end face image contour; obtaining a reference flange end face image, and at the same time, obtaining the target area of the current flange end face image, adjusting the reference flange end face image according to the target area, and obtaining a standard flange end face image; collecting the target coordinate values of the edge points of the standard flange end face image in the plane rectangular coordinate system; mapping the current flange end face image The coordinate values of the edge points of the image contour are matched with the target coordinate values of the edge points of the standard flange end face image in the plane rectangular coordinate system to determine whether the current flange end face image contour is accurate; if the coordinate values of the edge points of the current flange end face image contour match the target coordinate values of the edge points of the standard flange end face image in the plane rectangular coordinate system, then it is determined whether the current flange end face image contour is accurate, and the current flange end face image contour is used as the final output flange end face image contour; otherwise, it is determined that the current flange end face image contour is inaccurate, and at the same time, the difference points in the edge points of the current flange end face image contour are located according to the edge points of the standard flange end face image, and the difference points are corrected according to the target coordinate values of the corresponding edge points in the standard flange end face image until the current flange end face image contour is accurate.

[0052] This embodiment also provides a flange surface defect classification system based on machine vision, such as Figure 3 As shown, it includes: an image acquisition module, a contour determination module and a defect classification module, wherein the image acquisition module acquires the flange end face image; the contour determination module determines the image contour of the flange end face based on the target algorithm; the defect classification module extracts the target Hu matrix features and trains to generate a support vector machine, and at the same time, the image contour of the flange end face is classified into defects according to the support vector machine.

[0053] Among them, the image acquisition module includes capturing the flange end face image according to the industrial camera, sending the obtained flange end face image to the computer system according to the USB interface, transmitting the flange end face image to the computer end according to the driver, and processing the image based on the computer end in an integrated development environment.

[0054] As mentioned above, in the denoising stage (image processing includes denoising), a bilateral filtering algorithm is used instead of the traditional Gaussian filter to effectively suppress image noise while preserving edge information, thereby improving the algorithm's performance during the filtering process. In the gradient calculation stage, although the Sobel operator continues to be used as the core calculation method, it is expanded on this basis to add detection templates in the 45° and 135° directions to enrich the extraction of edge information and enhance the comprehensiveness of edge detection. In the threshold determination process, the Otsu algorithm is introduced to automatically determine the adaptive dual threshold, effectively improving the accuracy and robustness of edge detection. After completing edge detection, the improved Hu rectangular feature is further extracted. While ensuring scale invariance, it also satisfies rotation and translation invariance, providing more stable feature support for subsequent defect classification. Finally, through the performance comparison of four kernel functions, the polynomial kernel function is selected for training the SVM, forming the improved CannyH-SVM algorithm, which effectively improves the accuracy and robustness of edge detection. The target Hu matrix features effectively provide more stable feature support. By accurately training the support vector machine, it is conducive to accurate defect classification of flange labels.

[0055] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A flange surface defect classification method based on machine vision, characterized in that: include: Step 1: Collect the flange end face image; Step 2: Determine the image contour of the flange end face based on the target algorithm; Step 3: Extract the target Hu matrix features and train them to generate a support vector machine. At the same time, classify the defects of the image contour of the flange end face based on the support vector machine.

2. The flange surface defect classification method based on machine vision according to claim 1 is characterized in that: In step 1, the flange end face image is collected, including: The flange end face image is captured by an industrial camera, and the obtained flange end face image is sent to a computer system through a USB interface. The flange end face image is transmitted to the computer end according to a driver, and the image is processed based on the computer end in an integrated development environment.

3. The flange surface defect classification method based on machine vision according to claim 1 is characterized in that: In step 2, the target algorithm is the improved Canny algorithm, wherein the improved Canny algorithm uses bilateral filtering instead of Gaussian filtering for denoising and adds gradient calculations in the 45° and 135° directions.

4. The flange surface defect classification method based on machine vision according to claim 1 is characterized in that: In step 2, the image contour of the flange end face is determined based on the target algorithm, including: The flange end face image is pre-processed for denoising based on the target algorithm; The flange end face image after denoising is positioned, the edge points of the flange end face image are determined, and the edge points are connected to determine the image contour of the flange end face.

5. The flange surface defect classification method based on machine vision according to claim 3 or 4, characterized in that: When determining the edge point of the flange end face image, the first gradient intensity given by the horizontal defense line of the edge point and the second gradient intensity in the edge direction are obtained respectively; the first gradient intensity is compared with the second gradient intensity. When the first gradient intensity is greater than the second extraction intensity, the edge point of the flange end face image is determined to be the horizontal gradient value; otherwise, the vertical gradient value is selected.

6. The flange surface defect classification method based on machine vision according to claim 4 is characterized in that: The denoising process of the flange end face image includes: S101: performing a grayscale operation on the flange end surface image to obtain a grayscale image, and setting a filter window according to a preset ratio; S102: randomly selecting a central pixel in the grayscale image, and calculating the spatial distance weight between each pixel in the filter window and the central pixel; S103: Calculate the gray value similarity weight between each pixel in the filter window and the central pixel; S104: Multiplying the spatial distance weight corresponding to each pixel point and the central pixel point by the gray value similarity weight to obtain a comprehensive weight; S105: Calculating the grayscale value of the filtered central pixel according to the comprehensive weight; S106: Repeat steps S101 to S105 based on each pixel in the grayscale image to complete the image filtering operation.

7. The flange surface defect classification method based on machine vision according to claim 1 is characterized in that: In step 3, before extracting the target Hu matrix features, the order and scale factor of the normalized central moment in the original Hu matrix are adjusted to obtain the target Hu matrix.

8. The flange surface defect classification method based on machine vision according to claim 1 is characterized in that: Step 3 includes: collecting a preset number of sample images, extracting Hu moment features corresponding to the target Hu matrix in the sample images, and training the Hu moment features according to a polynomial kernel function to obtain a support vector machine; and classifying defects of the image contour of the flange end face according to the support vector machine.

9. A flange surface defect classification system based on machine vision, characterized in that: include: Image acquisition module, contour determination module and defect classification module, among which the image acquisition module acquires the flange end face image; the contour determination module determines the image contour of the flange end face based on the target algorithm; the defect classification module extracts the target Hu matrix features and trains to generate a support vector machine. At the same time, the image contour of the flange end face is classified into defects based on the support vector machine.

10. The flange surface defect classification system based on machine vision according to claim 9, characterized in that: The image acquisition module includes capturing the flange end face image according to the industrial camera, sending the obtained flange end face image to the computer system according to the USB interface, transmitting the flange end face image to the computer end according to the driver, and processing the image based on the computer end in an integrated development environment.

Images