Defect detection method of building structure steel plate based on machine vision

Abstract

The present invention relates to the technical field of image threshold segmentation, and in particular to a method for detecting defects in building structure steel plates based on machine vision, the method comprising: obtaining a target surface image corresponding to a building structure steel plate to be detected, and connecting all pixel points at each grayscale value in the target surface image; performing scratch defect shape regularity analysis and processing on each connected area; determining a target defect index at each grayscale value in the target surface image; determining the target information entropy of the target surface image at each preset segmentation threshold according to the grayscale value in the target surface image and the target defect index thereunder; screening out a preset segmentation threshold with the maximum target information entropy; performing threshold segmentation on the target surface image according to the target segmentation threshold, and judging whether each target segmentation area is a scratch defect area. The present invention improves the efficiency of scratch defect detection on building structure steel plates by performing adaptive threshold segmentation on the target surface image.

Description

Technical Field

[0001] The invention relates to the technical field of image threshold segmentation, and in particular to a method for detecting defects in building structure steel plates based on machine vision. Background Art

[0002] Building structural steel plates are steel plates used in building structures and are an important part of buildings. Defect detection can help to timely discover quality problems of steel plates and deal with them, extend the service life of steel plates, and improve the overall quality of buildings. Therefore, defect detection of building structural steel plates is very important. Among them, scratch defects are a common defect in building structural steel plates.

[0003] The existence of scratch defects often weakens the strength of steel plate structural materials, and scratch defects are easy to become the starting point of cracks when used in construction; secondly, scratches are often more susceptible to corrosion, which makes the surface and internal structure of the steel plate material more susceptible to severe corrosion, and then makes the structural hardness of the steel plate significantly weaker, resulting in a serious decrease in the load bearing capacity of the steel plate for the building, making it more likely to break, affecting the quality of the building. Therefore, it is very important to detect scratch defects on building structural steel plates.

[0004] At present, when performing defect detection on an object, a method usually adopted is to perform defect detection on the object through a neural network.

[0005] However, when using neural networks to detect scratch defects on building structural steel plates, the following technical problems often occur:

[0006] When training a neural network, a large number of images of building structure steel plates with scratch defects and images of building structure steel plates without scratch defects are often required. The collection of these images often takes a lot of time, and the time for training the neural network is often also long. Therefore, it often leads to poor efficiency in scratch defect detection of building structure steel plates. Summary of the invention

[0007] In order to solve the technical problem of low efficiency in detecting scratch defects on building structure steel plates, the present invention proposes a method for detecting defects on building structure steel plates based on machine vision.

[0008] The present invention provides a method for detecting defects in building structure steel plates based on machine vision, the method comprising:

[0009] Obtain a target surface image corresponding to the building structure steel plate to be detected, and connect all pixel points at each grayscale value in the target surface image to obtain a connected area at each grayscale value;

[0010] According to the fitting straight line of each connected area at each gray value and the straight line where the long side of its minimum circumscribed rectangle is located, as well as the gradient direction corresponding to all edge pixels in each connected area, each connected area is analyzed and processed for the shape law of scratch defects, and the defect manifestation factor corresponding to each connected area is obtained;

[0011] Determine the target defect index at each gray value based on the defect manifestation factors corresponding to all connected areas at each gray value and the gradient directions corresponding to all edge pixels in all connected areas at each gray value;

[0012] Determine the target information entropy of the target surface image at each preset segmentation threshold according to the grayscale values ​​corresponding to all pixels in the target surface image and the target defect indicators at all grayscale values ​​in the target surface image;

[0013] Select a preset segmentation threshold with the largest target information entropy from all preset segmentation thresholds as the target segmentation threshold;

[0014] According to the target segmentation threshold, the target surface image is segmented by threshold to obtain target segmentation areas, and it is determined whether each target segmentation area is a scratch defect area.

[0015] Optionally, the method performs scratch defect shape regularity analysis on each connected area according to the fitting straight line of each connected area at each gray value and the straight line where the long side of its minimum circumscribed rectangle is located, and the gradient direction corresponding to all edge pixels in each connected area, to obtain the defect manifestation factor corresponding to each connected area, including:

[0016] Determine any gray value in the target surface image as a marked gray value, and determine any connected area under the marked gray value as a marked connected area;

[0017] Filtering out a target fitting straight line corresponding to the marked connected region from all the fitting straight lines of the marked connected region;

[0018] Determine the straight line where the long side of the minimum circumscribed rectangle of the marked connected region lies as the reference straight line corresponding to the marked connected region;

[0019] The angle between the target fitting straight line corresponding to the marked connected area and the reference straight line is determined as the target angle corresponding to the marked connected area;

[0020] Determining an initial defect factor corresponding to the marked connected area according to a target angle corresponding to the marked connected area;

[0021] A defect manifestation factor corresponding to the marked connected region is determined according to the gradient directions corresponding to all edge pixels in the marked connected region and the initial defect factor corresponding to the marked connected region.

[0022] Optionally, the step of selecting a target fitting straight line corresponding to the marked connected region from all the fitting straight lines of the marked connected region includes:

[0023] According to the positions of all the pixels in the marked connected area, a preset number of different straight line fitting algorithms are used to perform straight line fitting on all the pixels in the marked connected area, so as to obtain the fitting straight line and the correction determination coefficient of the marked connected area under each straight line fitting algorithm, thereby forming the straight line fitting information of the marked connected area under each straight line fitting algorithm;

[0024] Screening out the straight line fitting information with the largest correction determination coefficient from the straight line fitting information of the marked connected area under a preset number of straight line fitting algorithms as the target fitting information corresponding to the marked connected area;

[0025] The fitting straight line included in the target fitting information corresponding to the marked connected region is determined as the target fitting straight line corresponding to the marked connected region.

[0026] Optionally, determining the defect manifestation factor corresponding to the marked connected region according to the gradient directions corresponding to all edge pixels in the marked connected region and the initial defect factor corresponding to the marked connected region includes:

[0027] The defect manifestation factor corresponding to the marked connected area is determined according to the variance of the gradient direction corresponding to all edge pixels in the marked connected area, the mean of the gradient direction corresponding to all edge pixels in the marked connected area, the angle corresponding to the reference straight line corresponding to the marked connected area, and the initial defect factor corresponding to the marked connected area.

[0028] Optionally, the formula corresponding to the defect representation factor corresponding to the connected area is:

[0029]

[0030] β ij =|Eθ ij -k ij |; Among them, P ij2 is the defect representation factor corresponding to the jth connected area at the i-th gray value in the target surface image; i is the serial number obtained by arranging the gray values ​​in the target surface image in ascending order; j is the serial number of the connected area at the i-th gray value in the target surface image; P ij1is the initial defect factor corresponding to the jth connected region at the i-th gray value in the target surface image; σ ij is the variance of the gradient direction corresponding to all edge pixels in the jth connected region at the i-th grayscale value in the target surface image; γ1 and γ2 are pre-set factors greater than 0; || is the absolute value function; β ij is the directional angle difference corresponding to the jth connected area under the i-th gray value in the target surface image; π is the radian value of 180°; n ij1 is the number of pixels in the jth connected region at the i-th grayscale value in the target surface image; n ij2 is the number of edge pixels in the jth connected region at the i-th grayscale value in the target surface image; R ij is the correction determination coefficient of the target fitting information corresponding to the jth connected area at the i-th gray value in the target surface image; exp() is an exponential function with a natural constant as the base; θ ij is the target angle corresponding to the jth connected area at the i-th gray value in the target surface image; Eθ ij is the mean value of the gradient direction corresponding to all edge pixels in the jth connected region under the i-th gray value in the target surface image; k ij It is the angle of the reference straight line corresponding to the jth connected area at the i-th gray value in the target surface image.

[0031] Optionally, determining the target defect index at each gray value according to the defect manifestation factors corresponding to all connected regions at each gray value and the gradient directions corresponding to all edge pixels in all connected regions at each gray value includes:

[0032] The target defect index at each gray value is determined based on the defect manifestation factors corresponding to all connected areas at each gray value and the mean of the gradient directions corresponding to all edge pixels in each connected area at each gray value.

[0033] Optionally, the formula corresponding to the target defect index under the gray value in the target surface image is:

[0034] Among them, YZ i is the target defect index at the i-th grayscale value in the target surface image; i is the serial number obtained by arranging the grayscale values ​​in the target surface image in ascending order; N i is the number of connected regions at the i-th gray value in the target surface image; j is the serial number of the connected region at the i-th gray value in the target surface image; P ij2 is the defect representation factor corresponding to the jth connected region at the i-th gray value in the target surface image; n ij1is the number of pixels in the jth connected region at the i-th grayscale value in the target surface image; n i is the number of pixels in the target surface image whose corresponding grayscale value is equal to the i-th grayscale value; S ij is the similarity between the jth connected region at the i-th gray value in the target surface image and the reference region at the i-th gray value; the reference region at the i-th gray value is the connected region with the largest number of pixels at the i-th gray value; || is the absolute value function; Eθ ij It is the mean value of the gradient direction corresponding to all edge pixels in the jth connected area under the i-th gray value in the target surface image; Eθ i is the mean value of the gradient direction corresponding to all edge pixels in the reference area at the i-th grayscale value in the target surface image; γ3 and γ4 are pre-set factors greater than 0; P i2 It is the defect manifestation factor corresponding to the reference area at the i-th gray value in the target surface image.

[0035] Optionally, determining the target information entropy of the target surface image at each preset segmentation threshold according to the grayscale values ​​corresponding to all pixels in the target surface image and the target defect indicators at all grayscale values ​​in the target surface image includes:

[0036] Determine the proportion of pixels at each grayscale value in the target surface image according to the grayscale values ​​corresponding to all pixels in the target surface image;

[0037] According to the proportion of pixels at all gray values ​​in the target surface image and the target defect index at all gray values ​​in the target surface image, the target information entropy of the target surface image at each preset segmentation threshold is determined.

[0038] Optionally, the formula corresponding to the target information entropy of the target surface image under a preset segmentation threshold is:

[0039] H t =H t1 +H t2 ;

[0040]

[0041] Among them, H t is the target information entropy of the target surface image under the preset segmentation threshold equal to the tth gray value; t, i and b are the serial numbers of the gray values ​​in the target surface image arranged in ascending order; H t1 is the foreground information entropy of the target surface image at the preset segmentation threshold equal to the t-th gray value; H t2is the background information entropy of the target surface image at the preset segmentation threshold equal to the t-th gray value; Q b is the defect feature at the bth gray value in the target surface image; n b is the number of pixels in the target surface image whose corresponding grayscale value is equal to the bth grayscale value; n is the number of pixels in the target surface image; The base is 2 The logarithm of; B is the maximum number obtained by arranging the gray values ​​in the target surface image in ascending order; YZ b YZ is the target defect index at the bth gray value in the target surface image; i is the target defect index at the i-th gray value in the target surface image; It is the proportion of pixels with the bth grayscale value in the target surface image.

[0042] Optionally, the determining whether each target segmented area is a scratch defect area includes:

[0043] Normalize the accumulated values ​​of the target defect indicators at all gray values ​​in each target segmentation area to obtain the scratch defect characteristic factor corresponding to each target segmentation area;

[0044] When the scratch defect characteristic factor corresponding to the target segmentation region is greater than a preset defect threshold, the target segmentation region is determined as a scratch defect region.

[0045] The present invention has the following beneficial effects:

[0046] The machine vision-based building structure steel plate defect detection method of the present invention solves the technical problem of poor efficiency of scratch defect detection on building structure steel plates by processing the target surface image, and improves the efficiency of scratch defect detection on building structure steel plates. First, since the target surface image contains the surface information of the building structure steel plate to be detected, obtaining the target surface image corresponding to the building structure steel plate to be detected can facilitate the subsequent scratch defect detection of the building structure steel plate to be detected. Secondly, obtaining the connected area under each gray value can facilitate the subsequent analysis and processing of each connected area. Next, since the shape of the scratch defect is often a relatively long and narrow rectangle, if a certain gray value is the gray value in the scratch defect, the connected area under the gray value often presents a long and narrow shape, and the fitting straight line of the connected area under the gray value is often parallel to the straight line where the long side of its minimum circumscribed rectangle is located. Therefore, the connected area is analyzed and processed for the shape law of the scratch defect, and the defect performance factor corresponding to the quantified connected area can characterize the defect situation of the connected area. The larger the value, the more likely it is that the connected area is located in the scratch defect area. Then, based on the defect manifestation factors corresponding to all connected areas under the gray value, and the gradient directions corresponding to all edge pixels in all connected areas under the gray value, the larger the quantized target defect index under the gray value, the more likely it is that the gray value is a gray value in the scratch defect area. Afterwards, based on the gray values ​​corresponding to all pixels in the target surface image, and the target defect index under all gray values ​​in the target surface image, the larger the target information entropy of the quantized target surface image under the preset segmentation threshold, the better the effect of threshold segmentation of the target surface image using the preset segmentation threshold. And compared with only considering the gray values ​​corresponding to the pixels in the target surface image to determine the information entropy, the present invention also considers the target defect index under all gray values ​​in the target surface image, which improves the accuracy of determining the target information entropy to a certain extent, thereby improving the accuracy of the target segmentation threshold screening, and further improving the accuracy of scratch defect detection on the building structure steel plate. Then, the preset segmentation threshold with the largest screened target information entropy is often the preset segmentation threshold with the best threshold segmentation effect among all preset segmentation thresholds. Finally, based on the target segmentation threshold, the target surface image is threshold segmented to achieve scratch defect segmentation, and it is judged whether each target segmentation area is a scratch defect area, thereby achieving scratch defect detection. In addition, the present invention does not need to collect a large number of images for training the neural network, nor does it need to spend time on training the neural network. Therefore, the present invention improves the efficiency of scratch defect detection on building structure steel plates. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 The present invention is a flow chart of the method for detecting defects in building structure steel plates based on machine vision. DETAILED DESCRIPTION

[0049] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0050] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0051] refer to Figure 1 , shows the process of some embodiments of the method for detecting defects in building structure steel plates based on machine vision according to the present invention. The method for detecting defects in building structure steel plates based on machine vision comprises the following steps:

[0052] Step S1, obtaining a target surface image corresponding to the building structure steel plate to be detected, and connecting all pixel points at each gray value in the target surface image to obtain a connected area at each gray value.

[0053] Among them, the building structure steel plate to be inspected can be a steel plate used in a building structure to be inspected for scratch defects. The building structure steel plate may be subjected to mechanical effects such as cutting, scratching and collision during processing and transportation, which may cause scratch defects on the building structure steel plate. The steel plate is a flat steel material formed by pouring molten steel and pressing it after cooling. It is flat and rectangular and can be directly rolled or cut from a wide steel strip. The target surface image can be a grayscale surface image of the building structure steel plate to be inspected. All pixel points under the grayscale value in the target surface image, that is, all pixel points in the target surface image that are equal to the grayscale value. The connected area under the grayscale value can be an area composed of pixel points in the target surface image that are equal to the grayscale value and connected together.

[0054] It should be noted that, since the target surface image contains the surface information of the building structure steel plate to be detected, obtaining the target surface image corresponding to the building structure steel plate to be detected can facilitate the subsequent scratch defect detection of the building structure steel plate to be detected. Secondly, obtaining the connected area under each gray value can facilitate the subsequent analysis and processing of each connected area.

[0055] As an example, this step may include the following steps:

[0056] The first step is to obtain the target surface image corresponding to the building structure steel plate to be inspected.

[0057] For example, a surface image of a steel plate of a building structure to be inspected may be captured by a CCD (Charge-Coupled Device) camera as an initial surface image, the initial surface image may be grayscaled, and the grayscaled initial surface image may be used as a target surface image.

[0058] In the second step, all pixels at each grayscale value in the target surface image are connected to obtain the connected area at each grayscale value.

[0059] For example, any gray value in the target surface image can be determined as the marked gray value, and a connected domain search is performed on the pixel points in the target surface image whose corresponding gray value is equal to the marked gray value through a connected domain search algorithm, and each connected domain in the area formed by all the pixel points in the target surface image whose corresponding gray value is equal to the marked gray value is used as a connected area under the marked gray value. The connected domain search algorithm can be a DFS (Deep First Search) algorithm.

[0060] Step S2, according to the fitting straight line of each connected area at each gray value and the straight line where the long side of its minimum circumscribed rectangle lies, as well as the gradient direction corresponding to all edge pixels in each connected area, each connected area is analyzed and processed for the shape regularity of the scratch defect to obtain the defect manifestation factor corresponding to each connected area.

[0061] The gradient direction corresponding to a pixel point refers to the direction in which the grayscale of the image at the pixel point changes fastest. The gradient direction can be represented by an angle. For example, if the gradient direction is horizontal, the gradient direction can be represented by 0°. Secondly, the "gradient direction" mentioned in the embodiments of the present invention, unless otherwise specified, represents an angle.

[0062] It should be noted that since the shape of a scratch defect is often a relatively long and narrow rectangle, if a certain grayscale value is the grayscale value of a scratch defect, the connected area under this grayscale value is often long and narrow, and the fitting straight line of the connected area under this grayscale value is often parallel to the straight line where the long side of its minimum circumscribed rectangle is located. Therefore, the connected area is analyzed and processed for the shape regularity of the scratch defect, and the defect expression factor corresponding to the quantified connected area can characterize the defect situation of the connected area. The larger the value, the more likely it is that the connected area is located in the scratch defect area.

[0063] As an example, this step may include the following steps:

[0064] In the first step, any gray value in the target surface image is determined as a marked gray value, and any connected area under the marked gray value is determined as a marked connected area.

[0065] The second step is to select the target fitting straight line corresponding to the marked connected region from all the fitting straight lines of the marked connected region, which may include the following sub-steps:

[0066] In the first sub-step, according to the positions of all pixel points in the above-mentioned marked connected area, a preset number of different straight-line fitting algorithms are used to perform straight-line fitting on all pixel points in the above-mentioned marked connected area, and the fitting straight line and correction determination coefficient of the above-mentioned marked connected area under each straight-line fitting algorithm are obtained, which constitute the straight-line fitting information of the above-mentioned marked connected area under each straight-line fitting algorithm.

[0067] Among them, the preset number can be a preset number. For example, the preset number can be 5. The straight line fitting algorithm can be an algorithm for straight line fitting. For example, the straight line fitting algorithm can be but is not limited to: least squares method, gradient descent method, stochastic gradient descent method and regularized linear regression. The correction determination coefficient is used to evaluate the quality of straight line fitting. The correction determination coefficient is also called the correction determination coefficient or determination coefficient. The method for obtaining the fitting straight line of the marked connected area under a certain straight line fitting algorithm can be: through the straight line fitting algorithm, the pixel points in the marked connected area are fitted with a straight line, and the fitting straight line obtained is the fitting straight line of the marked connected area under the straight line fitting algorithm. The method for obtaining the correction determination coefficient of the marked connected area under a certain straight line fitting algorithm can be: through the straight line fitting algorithm, the pixel points in the marked connected area are fitted with a straight line, and the correction determination coefficient calculated during the straight line fitting at this time is the correction determination coefficient of the marked connected area under the straight line fitting algorithm. The straight line fitting information of the marked connected area under a certain straight line fitting algorithm may include: the correction determination coefficient and the fitting straight line of the marked connected area under the straight line fitting algorithm.

[0068] In the second sub-step, the straight line fitting information with the largest correction determination coefficient is selected from the straight line fitting information of the above-mentioned marked connected area under a preset number of straight line fitting algorithms as the target fitting information corresponding to the above-mentioned marked connected area.

[0069] In a third sub-step, the fitting straight line included in the target fitting information corresponding to the marked connected region is determined as the target fitting straight line corresponding to the marked connected region.

[0070] In the third step, the straight line where the long side of the minimum circumscribed rectangle of the marked connected area lies is determined as the reference straight line corresponding to the marked connected area.

[0071] In the fourth step, the angle between the target fitting straight line corresponding to the marked connected area and the reference straight line is determined as the target angle corresponding to the marked connected area.

[0072] The target angle may range from 0° to 90°.

[0073] The fifth step is to determine the initial defect factor corresponding to the marked connected area according to the target angle corresponding to the marked connected area.

[0074] In the sixth step, the defect manifestation factor corresponding to the marked connected region is determined according to the gradient directions corresponding to all edge pixels in the marked connected region and the initial defect factor corresponding to the marked connected region.

[0075] For example, the defect manifestation factor corresponding to the marked connected area can be determined based on the variance of the gradient direction corresponding to all edge pixels in the marked connected area, the mean of the gradient direction corresponding to all edge pixels in the marked connected area, the angle corresponding to the reference straight line corresponding to the marked connected area, and the initial defect factor corresponding to the marked connected area.

[0076] For example, the formula corresponding to the defect representation factor of the connected area can be:

[0077]

[0078] β ij =|Eθ ij -k ij |; Among them, P ij2 is the defect representation factor corresponding to the jth connected area at the i-th gray value in the target surface image. i is the serial number obtained by arranging the gray values ​​in the target surface image in ascending order. j is the serial number of the connected area at the i-th gray value in the target surface image. ij1 is the initial defect factor corresponding to the jth connected region at the i-th gray value in the target surface image. ijis the variance of the gradient direction corresponding to all edge pixels in the jth connected region at the i-th grayscale value in the target surface image. γ1 and γ2 are pre-set factors greater than 0, mainly used to prevent the denominator from being 0, such as γ1 and γ2 can be 0.001. || is the absolute value function. β ij is the direction angle difference corresponding to the jth connected area at the i-th gray value in the target surface image. π is the radian value of 180°. n ij1 is the number of pixels in the jth connected region at the i-th grayscale value in the target surface image. ij2 R is the number of edge pixels in the jth connected region at the i-th grayscale value in the target surface image. ij is the correction determination coefficient of the target fitting information corresponding to the jth connected area at the i-th gray value in the target surface image. exp() is an exponential function with a natural constant as the base. θ ij It is the target angle corresponding to the jth connected area at the i-th gray value in the target surface image. ij k is the mean value of the gradient direction corresponding to all edge pixels in the jth connected region under the i-th grayscale value in the target surface image. ij It is the angle of the reference line corresponding to the jth connected area under the i-th gray value in the target surface image. The angle setting method of the reference line can refer to the setting method of the gradient direction. For example, if the gradient direction is horizontal, the gradient direction can be represented by 0°; if a reference line is a horizontal line, the angle corresponding to the reference line can be 0°.

[0079] It should be noted that when When R is larger, it often means that the number of edge pixels in the jth connected region under the i-th gray value in the target surface image is closer to the number of pixels inside it, and the jth connected region under the i-th gray value is more narrow and long, and the jth connected region under the i-th gray value is more linear, and the jth connected region under the i-th gray value is more likely to be within the scratch defect. ij can be used as exp(-θ ij ) weight, and when R ij The larger the value, the better the fitting effect of the straight line fitting for the jth connected area under the i-th gray value. ijThe smaller it is, the smaller the target angle corresponding to the jth connected area at the i-th gray value in the target surface image is, the smaller the angle between the target fitting line corresponding to the jth connected area at the i-th gray value and the reference line is, the more likely the target fitting line of the jth connected area at the i-th gray value is to be parallel to the long side of its minimum circumscribed rectangle, the more linear the jth connected area at the i-th gray value is, and the more likely the jth connected area at the i-th gray value is to be within the scratch defect. Therefore, when P ij1 When σ is larger, it often means that the jth connected area under the ith gray value is more linear, and it often means that the jth connected area under the ith gray value is more likely to be within the scratch defect. Generally speaking, the gradient direction distribution corresponding to the edge pixels of the narrow and long area is relatively uniform, that is, the gradient directions corresponding to most edge pixels of the narrow and long area are similar. ij The smaller it is, the more consistent the distribution of the gradient direction corresponding to the edge pixel points in the jth connected region under the i-th grayscale value in the target surface image is. This often means that the jth connected region under the i-th grayscale value is more likely to be a narrow and long region. This often means that the jth connected region under the i-th grayscale value is more likely to be a region within a scratch defect. Generally speaking, the straight line corresponding to the gradient direction of the edge pixel points in a narrow and long region is often perpendicular to the narrow and long direction of the narrow and long region. ij The closer When P is , it often means that the straight line corresponding to the gradient direction of the edge pixel point in the jth connected area at the i-th gray value in the target surface image is more likely to be perpendicular to the straight line of the long side of the minimum circumscribed rectangle of the connected area; it often means that the jth connected area at the i-th gray value is more likely to be a narrow and long area, and it often means that the jth connected area at the i-th gray value is more likely to be an area within a scratch defect. Therefore, when P ij2 When it is larger, it often means that the jth connected area under the i-th gray value in the target surface image is more likely to be the area within the scratch defect.

[0080] Step S3, determining the target defect index at each gray value according to the defect manifestation factors corresponding to all connected areas at each gray value and the gradient directions corresponding to all edge pixels in all connected areas at each gray value.

[0081] It should be noted that based on the defect manifestation factors corresponding to all connected areas under the grayscale value and the gradient directions corresponding to all edge pixels in all connected areas under the grayscale value, the larger the quantified target defect index under the grayscale value, the more likely it is that the grayscale value is a grayscale value within the scratch defect area.

[0082] As an example, the target defect index at each gray value can be determined based on the defect manifestation factors corresponding to all connected areas at each gray value and the mean of the gradient directions corresponding to all edge pixels in each connected area at each gray value.

[0083] For example, the formula corresponding to the target defect index under the gray value in the target surface image can be:

[0084] Among them, YZ i is the target defect index at the i-th grayscale value in the target surface image. i is the serial number obtained by arranging the grayscale values ​​in the target surface image in ascending order. N i is the number of connected regions at the i-th grayscale value in the target surface image. j is the serial number of the connected region at the i-th grayscale value in the target surface image. ij2 It is the defect manifestation factor corresponding to the jth connected area at the i-th gray value in the target surface image. ij1 is the number of pixels in the jth connected region at the i-th grayscale value in the target surface image. i is the number of pixels in the target surface image whose corresponding grayscale value is equal to the i-th grayscale value. ij It is the similarity between the jth connected region at the i-th gray value in the target surface image and the reference region at the i-th gray value; the reference region at the i-th gray value is the connected region with the largest number of pixels at the i-th gray value. || is the absolute value function. Eθ ij It is the mean value of the gradient direction corresponding to all edge pixels in the jth connected region under the i-th grayscale value in the target surface image. i It is the mean value of the gradient direction corresponding to all edge pixels in the reference area at the i-th gray value in the target surface image. γ3 and γ4 are pre-set factors greater than 0, mainly used to prevent the denominator from being 0. For example, γ3 and γ4 can be 0.001. i2 It is the defect manifestation factor corresponding to the reference area at the i-th gray value in the target surface image.

[0085] It should be noted that when P ij2 When |Eθ is larger, it often means that the jth connected area under the i-th gray value in the target surface image is more likely to be an area within the scratch defect, and it often means that the i-th gray value in the target surface image is more likely to be a gray value within the scratch defect. Since the reference area under the i-th gray value is the connected area with the largest number of pixels under the i-th gray value, the reference area under the i-th gray value is often a more representative connected area under the i-th gray value, which often means that the reference area under the i-th gray value can relatively better represent the approximate characteristics of the connected area under the i-th gray value.ij -Eθ i The smaller | is, the closer the gradient direction of the jth connected region under the i-th gray value in the target surface image is to that of the edge pixels in its reference region, which often means that the jth connected region under the i-th gray value is more similar to its reference region. ij2 -P i2 The smaller | is, the closer the defect expression factor of the jth connected area under the i-th gray value in the target surface image is to that of the reference area, and the more similar the scratch defect situation of the jth connected area under the i-th gray value is to that of the reference area. ij It can characterize the similarity between the jth connected region at the ith gray value in the target surface image and its reference region. The larger its value is, the more similar the jth connected region at the ith gray value in the target surface image is to its reference region, and the more consistent the jth connected region at the ith gray value in the target surface image is with the approximate characteristics of the connected region at the ith gray value. Can be used as S ij Therefore, when YZ i The larger it is, the more likely it is that the i-th grayscale value in the target surface image is a grayscale value within the scratch defect.

[0086] Step S4, determining the target information entropy of the target surface image at each preset segmentation threshold according to the grayscale values ​​corresponding to all pixels in the target surface image and the target defect indicators at all grayscale values ​​in the target surface image.

[0087] It should be noted that, based on the grayscale values ​​corresponding to all pixels in the target surface image and the target defect indicators at all grayscale values ​​in the target surface image, the larger the target information entropy of the quantized target surface image under the preset segmentation threshold, the better the effect of threshold segmentation of the target surface image using the preset segmentation threshold. And compared with only considering the grayscale values ​​corresponding to the pixels in the target surface image to determine the information entropy, the present invention also considers the target defect indicators at all grayscale values ​​in the target surface image, which improves the accuracy of the target information entropy determination to a certain extent.

[0088] As an example, this step may include the following steps:

[0089] The first step is to determine the proportion of pixels at each grayscale value in the target surface image according to the grayscale values ​​corresponding to all pixels in the target surface image.

[0090] In the second step, according to the proportion of pixels at all gray values ​​in the target surface image and the target defect index at all gray values ​​in the target surface image, the target information entropy of the target surface image at each preset segmentation threshold is determined.

[0091] For example, the formula corresponding to the target information entropy of the target surface image under the preset segmentation threshold can be:

[0092] H t =H t1 +H t2 ;

[0093]

[0094] Among them, H t is the target information entropy of the target surface image at the preset segmentation threshold equal to the tth gray value. t, i and b are the serial numbers of the gray values ​​in the target surface image arranged in ascending order. t1 It is the foreground information entropy of the target surface image at the preset segmentation threshold equal to the tth gray value. t2 Q is the background information entropy of the target surface image at the preset segmentation threshold equal to the tth gray value. b is the defect feature at the bth gray value in the target surface image. b is the number of pixels in the target surface image whose corresponding grayscale value is equal to the bth grayscale value. n is the number of pixels in the target surface image. The base is 2 B is the maximum number of gray values ​​in the target surface image arranged in ascending order, that is, the number of gray values ​​in the target surface image. b YZ is the target defect index at the bth gray value in the target surface image. i It is the target defect index at the i-th gray value in the target surface image. It is the proportion of pixels with the bth grayscale value in the target surface image.

[0095] It should be noted that when YZ b The larger the value, the more likely the bth gray value in the target surface image is to be a gray value within a scratch defect. Generally speaking, due to the influence of factors such as light, the gray value within the scratch defect area is often smaller than the gray value within the non-defect area. t1It can characterize the image entropy of the area composed of pixels whose grayscale value is less than or equal to the tth grayscale value in the target surface image, and can characterize the segmentation effect when the scratch defect is segmented from the target surface image. The larger its value is, the more accurate the segmented scratch defect area is. t1 Characterizes the segmentation effect when the scratch defect is segmented in the target surface image, so when calculating H t1 When Q, which represents the defect b As The weight of Characterizing the segmentation effect of scratch defects from the target surface image, the present invention adopts H t1 When characterizing the segmentation effect of the scratch defect in the target surface image, in addition to considering the grayscale chaos in the region, the scratch defect characteristics in the region are also considered. Therefore, H t1 To a certain extent, it can more accurately characterize the image entropy of the area composed of pixels whose grayscale values ​​are less than or equal to the tth grayscale value in the target surface image. t2 It can characterize the image entropy of the area composed of pixels with grayscale values ​​greater than the tth grayscale value in the target surface image, and can characterize the segmentation effect when non-scratch defects are segmented from the target surface image. The larger its value is, the more accurate the segmented non-scratch defect area is, and thus the more accurate the segmented scratch defect area is. t2 Characterizes the segmentation effect when the background area is segmented from the target surface image, where the background area is equivalent to the non-scratch defect area. t2 When the defect condition is characterized by 1-Q b As The weight of Characterizing the segmentation effect of non-scratch defects from the target surface image, the present invention adopts H t2 When characterizing the segmentation effect of non-scratch defects in the target surface image, in addition to considering the grayscale chaos in the region, the scratch defect characteristics in the region are also considered. Therefore, H t2 To a certain extent, it can more accurately characterize the image entropy of the area composed of pixels with grayscale values ​​greater than the tth grayscale value in the target surface image. t The larger it is, the more accurately the scratch defect segmentation of the target surface image can be achieved when the tth gray value in the target surface image is used as the segmentation threshold and the target surface image is segmented by threshold.

[0096] Step S5, selecting a preset segmentation threshold with the largest target information entropy from all preset segmentation thresholds as the target segmentation threshold.

[0097] It should be noted that the preset segmentation threshold with the largest screened target information entropy is often the preset segmentation threshold with the best threshold segmentation effect among all the preset segmentation thresholds.

[0098] As an example, a preset segmentation threshold with the maximum target information entropy may be used as the target segmentation threshold.

[0099] Step S6, performing threshold segmentation on the target surface image according to the target segmentation threshold to obtain target segmentation areas, and determining whether each target segmentation area is a scratch defect area.

[0100] It should be noted that based on the target segmentation threshold, the target surface image is threshold segmented to achieve scratch defect segmentation, and it is determined whether each target segmentation area is a scratch defect area to achieve scratch defect detection.

[0101] As an example, this step may include the following steps:

[0102] In the first step, according to the target segmentation threshold, the target surface image is threshold segmented to obtain the target segmentation area to achieve scratch defect segmentation.

[0103] For example, the area composed of pixels whose corresponding grayscale values ​​in the target surface image are less than or equal to the target segmentation threshold can be recorded as one target segmentation area; and the area composed of pixels whose corresponding grayscale values ​​in the target surface image are greater than the target segmentation threshold can be recorded as another target segmentation area, thus obtaining two target segmentation areas.

[0104] In the second step, the accumulated values ​​of the target defect indicators at all gray values ​​in each target segmentation area are normalized to obtain the scratch defect characteristic factor corresponding to each target segmentation area.

[0105] In the third step, when the scratch defect characteristic factor corresponding to the target segmentation area is greater than a preset defect threshold, the target segmentation area is determined as a scratch defect area.

[0106] The preset defect threshold may be a preset defect threshold, for example, the preset defect threshold may be 0.7.

[0107] In summary, compared with only considering the grayscale values ​​corresponding to the pixels in the target surface image to determine the information entropy, the present invention also considers the target defect indicators at all grayscale values ​​in the target surface image, which improves the accuracy of the target information entropy determination to a certain extent, thereby improving the accuracy of the target segmentation threshold screening, and further improving the accuracy of the scratch defect detection of the building structure steel plate. In addition, the present invention does not need to collect a large number of images for training the neural network, nor does it need to spend time training the neural network, so the present invention improves the efficiency of the scratch defect detection of the building structure steel plate.

[0108] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting defects in building structure steel plates based on machine vision, characterized in that: The following steps are involved: Obtain a target surface image corresponding to the building structure steel plate to be detected, and connect all pixel points at each grayscale value in the target surface image to obtain a connected area at each grayscale value; According to the fitting straight line of each connected area at each gray value and the straight line where the long side of its minimum circumscribed rectangle is located, as well as the gradient direction corresponding to all edge pixels in each connected area, each connected area is analyzed and processed for the shape law of scratch defects, and the defect manifestation factor corresponding to each connected area is obtained; Determine the target defect index at each gray value based on the defect manifestation factors corresponding to all connected areas at each gray value and the gradient directions corresponding to all edge pixels in all connected areas at each gray value; Determine the target information entropy of the target surface image at each preset segmentation threshold according to the grayscale values ​​corresponding to all pixels in the target surface image and the target defect indicators at all grayscale values ​​in the target surface image; Select a preset segmentation threshold with the largest target information entropy from all preset segmentation thresholds as the target segmentation threshold; According to the target segmentation threshold, the target surface image is threshold segmented to obtain target segmentation areas, and whether each target segmentation area is a scratch defect area is determined; According to the fitting straight line of each connected area at each gray value and the straight line where the long side of its minimum circumscribed rectangle is located, and the gradient direction corresponding to all edge pixels in each connected area, each connected area is subjected to a scratch defect shape regularity analysis process to obtain a defect manifestation factor corresponding to each connected area, including: Determine any gray value in the target surface image as a marked gray value, and determine any connected area under the marked gray value as a marked connected area; Filtering out a target fitting straight line corresponding to the marked connected region from all the fitting straight lines of the marked connected region; Determine the straight line where the long side of the minimum circumscribed rectangle of the marked connected region lies as the reference straight line corresponding to the marked connected region; The angle between the target fitting straight line corresponding to the marked connected area and the reference straight line is determined as the target angle corresponding to the marked connected area; Determining an initial defect factor corresponding to the marked connected area according to a target angle corresponding to the marked connected area; Determine a defect manifestation factor corresponding to the marked connected region according to the gradient directions corresponding to all edge pixels in the marked connected region and the initial defect factor corresponding to the marked connected region; The step of selecting a target fitting straight line corresponding to the marked connected region from all the fitting straight lines of the marked connected region includes: According to the positions of all the pixels in the marked connected area, a preset number of different straight line fitting algorithms are used to perform straight line fitting on all the pixels in the marked connected area, so as to obtain the fitting straight line and the correction determination coefficient of the marked connected area under each straight line fitting algorithm, thereby forming the straight line fitting information of the marked connected area under each straight line fitting algorithm; Screening out the straight line fitting information with the largest correction determination coefficient from the straight line fitting information of the marked connected area under a preset number of straight line fitting algorithms as the target fitting information corresponding to the marked connected area; Determine the fitting straight line included in the target fitting information corresponding to the marked connected region as the target fitting straight line corresponding to the marked connected region; The step of determining the defect manifestation factor corresponding to the marked connected region according to the gradient directions corresponding to all edge pixels in the marked connected region and the initial defect factor corresponding to the marked connected region comprises: Determine a defect manifestation factor corresponding to the marked connected area according to the variance of the gradient direction corresponding to all edge pixels in the marked connected area, the mean of the gradient direction corresponding to all edge pixels in the marked connected area, the angle corresponding to the reference straight line corresponding to the marked connected area, and the initial defect factor corresponding to the marked connected area; The formula for the defect manifestation factor corresponding to the connected area is: β ij =|Eθ ij -k ij |; Among them, P ij2 is the defect representation factor corresponding to the jth connected area at the i-th gray value in the target surface image; i is the serial number obtained by arranging the gray values ​​in the target surface image in ascending order; j is the serial number of the connected area at the i-th gray value in the target surface image; P ij1 is the initial defect factor corresponding to the jth connected region at the i-th gray value in the target surface image; σ ij is the variance of the gradient direction corresponding to all edge pixels in the jth connected region at the i-th grayscale value in the target surface image; γ1 and γ2 are pre-set factors greater than 0; || is the absolute value function; β ij is the directional angle difference corresponding to the jth connected area under the i-th gray value in the target surface image; π is the radian value of 180°; n ij1 is the number of pixels in the jth connected region at the i-th grayscale value in the target surface image; n ij2 is the number of edge pixels in the jth connected region at the i-th grayscale value in the target surface image; R ij is the correction determination coefficient of the target fitting information corresponding to the jth connected area at the i-th gray value in the target surface image; exp() is an exponential function with a natural constant as the base; θ ij is the target angle corresponding to the jth connected area at the i-th gray value in the target surface image; Eθ ij is the mean value of the gradient direction corresponding to all edge pixels in the jth connected region under the i-th gray value in the target surface image; k ij is the angle corresponding to the reference straight line corresponding to the jth connected area at the i-th gray value in the target surface image; Determining the target defect index at each gray value according to the defect manifestation factors corresponding to all connected areas at each gray value and the gradient directions corresponding to all edge pixels in all connected areas at each gray value includes: Determine the target defect index at each gray value based on the defect manifestation factors corresponding to all connected areas at each gray value and the mean of the gradient directions corresponding to all edge pixels in each connected area at each gray value; The formula corresponding to the target defect index under the gray value in the target surface image is: Among them, YZ i is the target defect index at the i-th grayscale value in the target surface image; i is the serial number obtained by arranging the grayscale values ​​in the target surface image in ascending order; N i is the number of connected regions at the i-th gray value in the target surface image; j is the serial number of the connected region at the i-th gray value in the target surface image; P ij2 is the defect representation factor corresponding to the jth connected region at the i-th gray value in the target surface image; n ij1 is the number of pixels in the jth connected region at the i-th grayscale value in the target surface image; n i is the number of pixels in the target surface image whose corresponding grayscale value is equal to the i-th grayscale value; S ij is the similarity between the jth connected region at the i-th gray value in the target surface image and the reference region at the i-th gray value; the reference region at the i-th gray value is the connected region with the largest number of pixels at the i-th gray value; || is the absolute value function; Eθ ij It is the mean value of the gradient direction corresponding to all edge pixels in the jth connected area under the i-th gray value in the target surface image; Eθ i is the mean value of the gradient direction corresponding to all edge pixels in the reference area at the i-th grayscale value in the target surface image; γ3 and γ4 are pre-set factors greater than 0; P i2 It is the defect manifestation factor corresponding to the reference area at the i-th gray value in the target surface image.

2. The method for detecting defects in building structure steel plates based on machine vision according to claim 1, characterized in that: Determining the target information entropy of the target surface image at each preset segmentation threshold according to the grayscale values ​​corresponding to all pixels in the target surface image and the target defect indicators at all grayscale values ​​in the target surface image includes: Determine the proportion of pixels at each grayscale value in the target surface image according to the grayscale values ​​corresponding to all pixels in the target surface image; According to the proportion of pixels at all gray values ​​in the target surface image and the target defect index at all gray values ​​in the target surface image, the target information entropy of the target surface image at each preset segmentation threshold is determined.

3. The method for detecting defects in building structure steel plates based on machine vision according to claim 2, characterized in that: The formula corresponding to the target information entropy of the target surface image under the preset segmentation threshold is: H t =H t1 +H t2 ; Among them, H t is the target information entropy of the target surface image under the preset segmentation threshold equal to the tth gray value; t, i and b are the serial numbers of the gray values ​​in the target surface image arranged in ascending order; H t1 is the foreground information entropy of the target surface image at the preset segmentation threshold equal to the t-th gray value; H t2 is the background information entropy of the target surface image at the preset segmentation threshold equal to the t-th gray value; Q b is the defect feature at the bth gray value in the target surface image; n b is the number of pixels in the target surface image whose corresponding grayscale value is equal to the bth grayscale value; n is the number of pixels in the target surface image; The base is 2 The logarithm of; B is the maximum number obtained by arranging the gray values ​​in the target surface image in ascending order; YZ b YZ is the target defect index at the bth gray value in the target surface image; i is the target defect index at the i-th gray value in the target surface image; It is the proportion of pixels with the bth grayscale value in the target surface image.

4. The method for detecting defects in building structure steel plates based on machine vision according to claim 1, characterized in that: The determining whether each target segmented area is a scratch defect area includes: Normalize the accumulated values ​​of the target defect indicators at all gray values ​​in each target segmentation area to obtain the scratch defect characteristic factor corresponding to each target segmentation area; When the scratch defect characteristic factor corresponding to the target segmentation region is greater than a preset defect threshold, the target segmentation region is determined as a scratch defect region.

Images