Metal surface rusty spot detection method based on machine vision

Abstract

The invention relates to the technical field of image processing, in particular to a metal surface rust spot detection method based on machine vision, which comprises the following steps: acquiring a sample image, and preprocessing the sample image; performing gray threshold calculation on the rusty spot area and the non-rusty spot area of the sample image by using a clustering algorithm; collecting a to-be-detected metal image, and obtaining a rusty spot region contour; judging whether a gray average value of pixel points in the rusty spot area is within an image gray threshold value or not; and calculating whether the proportion of the number of pixel points in the rusty spot area in the image gray threshold value exceeds a set threshold value or not. According to the method, the problem that a deep learning model is inaccurate in data set labeling and rusty spot region recognition is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for detecting rust spots on the metal surface by machine vision. Background Art

[0002] Rust spots on the metal surface are a common phenomenon that occurs during the use of metal devices due to the long-term action of environmental factors. It not only reduces the service life of metal devices, but also poses certain potential safety hazards to the use environment. Therefore, in order to timely deal with rust spots on the metal surface and improve the service life and safety of metal devices, the prerequisite is to be able to efficiently and accurately detect the area where rust spots on the metal surface are located.

[0003] In traditional methods for detecting rust spots on the metal surface, physical methods are generally used to detect certain characteristics of the metal, such as eddy current method and acoustic emission technology, etc. The eddy current method is to determine the position of rust spots on the surface of the metal to be tested by the change in the impedance of the detection coil when the detection coil with alternating current approaches the metal to be tested. The acoustic emission technology is to emit elastic waves through an acoustic emission source. When the elastic waves propagate to the surface of the metal to be tested, the position of rust spots on the surface of the metal to be tested is determined by converting the mechanical vibration of the metal to be tested into an electrical signal. However, these two methods are extremely susceptible to factors such as the complex environment of the industrial site and the shape of the metal, which reduces the detection efficiency and accuracy. Therefore, there is an urgent need for a method that can efficiently and accurately detect the area where rust spots on the metal surface are located.

[0004] For the patent with the publication number CN118840336A, the detection image of the inner wall of the corrugated pipe is obtained, and the detection image of the inner wall of the corrugated pipe is data-labeled to obtain a dataset of the detection image of the inner wall of the corrugated pipe. A deep learning model based on YOLOv5 is constructed. However, using a deep learning model requires a large amount of sample collection and data annotation, a large number of datasets need to be constructed, and the recognition of rust spots is through a detection frame. The detection frame is usually rectangular, while the area of rust spots is usually an irregular shape, resulting in inaccurate recognition of the rust spot area. Summary of the Invention

[0005] Aiming at the deficiencies of the existing methods, the present invention solves the problems of inaccurate dataset annotation and inaccurate recognition of rust spot areas in the deep learning model.

[0006] The technical solution adopted by the present invention is: a method for detecting rust spots on the metal surface by machine vision includes the following steps:

[0007] Step 1: Collect sample images and preprocess the sample images.

[0008] As a preferred embodiment of the present invention, the preprocessing includes grayscale change, image filtering, image enhancement, and image normalization.

[0009] Step 2: Use the clustering algorithm to calculate the gray - scale thresholds for the rusty areas and non - rusty areas of the sample image;

[0010] As a preferred embodiment of the present invention, the clustering algorithm is the kmeans algorithm.

[0011] As a preferred embodiment of the present invention, Step 2 specifically includes:

[0012] Step 21: Randomly select k pixel points on the sample image P 1 as the initial clustering center points c k , where k is the sum of the number of rusty areas and non - rusty areas on the image P 1 ;

[0013] Step 22: Calculate the distance 1 between each pixel point x i on the image P j and each initial clustering center point c and classify the pixel point x i into the cluster C j where the corresponding minimum - distance initial clustering center point c is located, to obtain a new cluster C' j ; k

[0014] Step 23: Calculate the coordinate mean c k of the pixel points x a within the new cluster C' b , and use the coordinate mean c b as the clustering center point of the new cluster C' k ;

[0015] Step 24: When the coordinate mean c k of the clustering center point of the new cluster C' b converges, stop the loop to obtain a new cluster C” k ;

[0016] Step 25: Take the minimum and maximum gray - scale values of the pixel points in the cluster C” k of the rusty area as the gray - scale threshold [T 1 , T 2 of the rusty image, and take the minimum and maximum gray - scale values of the pixel points in the cluster C” k of the non - rusty area as the gray - scale threshold [T 3 , T 4 of the non - rusty image;

[0017] Step 3: Collect the image of the metal to be detected and obtain the contour of the rusty area;

[0018] As a preferred embodiment of the present invention, the contour of the rusty area is obtained using OpenCV.​

[0019] As a preferred embodiment of the present invention, a convolutional network is also used to obtain the contour of the rusty area.

[0020] Step 4: Determine whether the gray scale mean value of the pixel points in the rusty area is within the image gray scale threshold;

[0021] As a preferred embodiment of the present invention, Step 4 specifically includes:

[0022] When the gray scale mean value T of the pixel points in the rusty area satisfies T 1 < T < T 2 or T < T 3 or T > T 4 , set the gray scale value of the pixel points in the rusty area to 0, otherwise set it to 255.

[0023] Step 5: Calculate whether the proportion of the number of pixel points in the rusty area within the image gray scale threshold to the number of pixel points in the rusty area exceeds a set threshold;

[0024] As a preferred embodiment of the present invention, Step 5 specifically includes:

[0025] When the proportion of the number of pixel points in the rusty area within the image gray scale threshold to the number of pixel points in the rusty area exceeds the set threshold, connect the coordinates of the outermost pixel points to obtain a circumscribed rectangle.

[0026] As a preferred embodiment of the present invention, a machine vision-based metal surface rust detection system includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement a machine vision-based metal surface rust detection method.

[0027] As a preferred embodiment of the present invention, a computer-readable medium storing computer program code, the computer program code implementing a machine vision-based metal surface rust detection method when executed by a processor.

[0028] Advantages of the present invention:

[0029] 1. The clustering analysis algorithm is used for unsupervised learning. By calculating the distances between each pixel point in the image and each clustering center point and comparing them, each cluster is classified, so as to obtain the gray scale threshold ranges of the rusty image and the non-rusty image, which can be effectively applied to complex industrial environments and improve the accuracy of metal surface rust detection;

[0030] 2. Taking the gray scale threshold range as the judgment criterion, the primary selection and final determination of the area where the metal surface rust is located are completed, which can effectively reduce the calculation amount in the subsequent image processing process and improve the efficiency of metal surface rust detection. Description of the Drawings

[0031] Figure 1 is the flowchart of the rust spot detection method for the metal surface by machine vision of the present invention;

[0032] Figure 2 is the schematic diagram of the rust spot area extracted by the image contour extraction algorithm;

[0033] Figure 3 is the schematic diagram of the rust spot area extracted by the method of the present invention. Specific Embodiments

[0034] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner. Therefore, it only shows the components related to the present invention.

[0035] As Figure 1 shown, a rust spot detection method for the metal surface by machine vision includes the following steps:

[0036] Step 1: Collect a sample image and preprocess the sample image;

[0037] The preprocessing includes grayscale change, image filtering, image enhancement, and image normalization;

[0038] The sample image is a metal device with rust spots on the metal surface, such as a metal pipe, a metal utensil, etc., which are devices prone to rust. Accurately obtaining the rusty area provides a basis for subsequent precise rust removal.

[0039] Step 2: Use the clustering algorithm to calculate the grayscale threshold of the rust spot area and the non-rust spot area of the sample image;

[0040] The clustering algorithm can adopt the kmeans algorithm, specifically including;

[0041] Step 21: First, randomly select k pixel points on the sample image P 1 as the initial clustering center points c k , c k indicating that there are k clusters C 1 in the image P k , c k ∈C k ; k is the sum of the number of rust spot areas and non-rust spot areas on the image P 1 ;

[0042] For example, in an image P 1 with 2 rust spots, if the number of non-rust spot areas is set to 1, the number of non-rust spot areas can also be set to other numbers; then, k = 3, c 1 , c 2 represent the points on the rust spot area, and c 3 represents the point on the non-rust spot area.

[0043] Step 22: Calculate the image P 1 for each pixel point x i on it and the distance to each initial clustering center point c j respectively and classify the pixel point x i into the cluster C j where the corresponding initial clustering center point c with the minimum distance is located; the formula for the distance j is as follows:

[0044]

[0045] where i ∈ (1, 2, …, n), j ∈ (1, 2, …, k), and n is the total number of pixel points on the image P 1 ;

[0046] The minimum distance means the minimum value of the distances between the pixel point x i and the k initial clustering center points c k respectively. For example, if the distance between the first pixel point x 1 of the image and c 1 is less than the distances to c 2 , c 3 , then x 1 is classified into the cluster C 1 ;

[0047] And so on, to obtain the new cluster C' k ;

[0048] Step 23: Calculate the coordinate mean c l of the pixel points x a within the new cluster C' b , and use the coordinate mean c b as the clustering center point of the new cluster C' k ; the formula for the coordinate mean is:

[0049]

[0050] where a ∈ (1, 2, …, m), b ∈ (1, 2, …, k), and m is the total number of pixel points within a certain new cluster C' k ;

[0051] Step 24: When the coordinate mean c l of the clustering center point of the new cluster C' b converges, stop the loop to obtain the new cluster C” k ; that is, the coordinate mean c b of the clustering center point no longer changes.

[0052] Step 25: The cluster C” k ​The minimum and maximum gray values of the pixels in the rust spot image are used as the gray threshold [T 1 ,T 2 , and the minimum and maximum gray values of the pixels in the cluster C” k of the non-rust spot area are used as the gray threshold [T 3 ,T 4 ;

[0053] Step 3: Collect the metal image to be detected and obtain the contour of the rust spot area;

[0054] The metal image to be detected needs to be preprocessed in the same way as the sample image, and the contour of the rust spot area is extracted using an image contour extraction algorithm;

[0055] The image contour extraction algorithm can use the OpenCV tool; or an image segmentation algorithm, such as the deep learning segmentation algorithm of a convolutional network.

[0056] As Figure 2 shown, the black area is the rust spot area. Since the contour acquisition of the rust spot area may include the non-rust spot area, resulting in inaccurate acquisition of the rust spot area and causing a certain deviation in subsequent rust spot processing, it is necessary to improve the recognition accuracy of the rust spot area.

[0057] Step 4: Determine whether the average gray value of the pixels in the rust spot area is within the image gray threshold. If it is satisfied, the pixel gray value is set to 0; otherwise, the pixel gray value is set to 255;

[0058] If the average gray value T of the pixels in the rust spot area satisfies the condition T 1 <T<T 2 or T<T 3 or T>T 4 , then the pixel is a pixel in the possible area of the rust spot on the metal surface, and the gray value of the pixel in the rust spot area is set to 0; otherwise, it is set to 255.

[0059] Step 5: Calculate whether the proportion of the number of pixels in the rust spot area within the image gray threshold to the number of pixels in the rust spot area exceeds the set threshold; if so, connect the coordinates of the outermost pixels that meet the conditions, and the obtained circumscribed rectangle is the final rust spot area;

[0060] If the proportion r of the number of pixels in the rust spot area within the image gray threshold to the number of pixels in the rust spot area is greater than the set threshold R, where R = 90%; then it is determined that the current contour is the final rust spot area; otherwise, the gray value of all pixels in the contour area is set to 255.

[0061] As Figure 3 shown, it is Figure 2 the corresponding final rust spot area.

[0062] Inspired by the above-described ideal embodiments of the present invention, through the above description, relevant staff can completely make various changes and modifications without departing from the technical idea of the present invention. The technical scope of the present invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for detecting rust on a metal surface using machine vision, characterized in that: It includes the following steps: Step 1: Collect a sample image and preprocess the sample image; Step 2: Use a clustering algorithm to calculate the gray - level threshold for the rusty area and non - rusty area of the sample image; Step 3: Collect the image of the metal to be detected and obtain the contour of the corresponding rusty area; Step 4: Determine whether the average gray - level value of the pixel points in the rusty area is within the image gray - level threshold; Step 5: Calculate whether the proportion of the number of pixel points in the rusty area within the image gray - level threshold to the number of pixel points in the rusty area exceeds a set threshold.

2. The method for detecting rust on a metal surface using machine vision according to claim 1, characterized in that: The clustering algorithm is the kmeans algorithm.

3. The method for detecting rust on a metal surface using machine vision according to claim 2, characterized in that: Step 2 specifically includes: Step 21: Randomly select k pixels on the sample image P1 as the initial cluster center point c k , k is the sum of the number of rust spots and the number of non-rust spots on image P1; Step 22: Calculate the x value for each pixel on image P1 i With each cluster initial center point c j Distance And the pixel x i Classify to the initial center point c of the cluster with the smallest corresponding distance j Cluster C j In the above example, we get a new cluster C' k ; Step 23: Calculate the new cluster C' k Inner pixel x a The coordinate mean c b , and the coordinate mean c b As a new cluster C' k The cluster center of Step 24: When the new cluster C' k The coordinate mean c of the cluster center b When convergence occurs, stop the loop and get a new cluster C" k ; Step 25: Cluster C" of the rusty area k The minimum and maximum grayscale values ​​of the pixels in the image are used as the grayscale threshold of the rust image [T1, T2], and the cluster C" in the non-rust area is k The minimum and maximum grayscale values ​​of the pixels in the image are taken as the grayscale thresholds of the non-rust image [T3, T4].

4. The method for detecting rust on a metal surface by machine vision according to claim 3, characterized in that: Step 4 specifically includes: When the average gray - level value T of the pixel points in the rusty area satisfies T1 < T < T2 or T < T3 or T > T4, set the gray - level value of the pixel points in the rusty area to 0, otherwise set it to 255.

5. The method for detecting rust on a metal surface by machine vision according to claim 1, characterized in that: Step 5 specifically includes: When the proportion of the number of pixel points in the rusty area within the image gray - level threshold to the number of pixel points in the rusty area exceeds the set threshold, connect the coordinates of the outermost pixel points to obtain a circumscribed rectangle.

6. The method for detecting rust on a metal surface by machine vision according to claim 1, characterized in that: The preprocessing includes gray - level change, image filtering, image enhancement, and image normalization.

7. The method for detecting rust on a metal surface by machine vision according to claim 1, characterized in that: The contour of the rusty area is obtained using OpenCV.

8. The method for detecting rust on a metal surface using machine vision according to claim 1, characterized in that: The contour of the rusty area is also obtained using a convolutional network.

9. The machine vision metal surface rust detection system is characterized by: It includes: A memory for storing instructions executable by a processor; A processor for executing the instructions to implement the method for detecting rust on the metal surface by machine vision according to any one of claims 1 - 8.

10. A computer readable medium storing computer program code, characterized in that: The computer program code implements the method for detecting rust on the metal surface by machine vision according to any one of claims 1 - 8 when executed by the processor.

Images