An intelligent judging method for surface quality of cold-rolled strip based on image processing

Abstract

This invention relates to the field of image processing technology and discloses an intelligent method for judging the surface quality of cold-rolled strip steel based on image processing. The method includes: acquiring bright-field and dark-field images of the strip steel surface, fusing the two images to obtain a preprocessed image; segmenting the preprocessed image to obtain the actual defect region; extracting the geometric features, texture features, and frequency domain features of the actual defect region to construct a multi-dimensional feature vector; identifying the defect category based on the multi-dimensional feature vector, reconstructing the three-dimensional morphology of the defect, and calculating the defect severity index; establishing a configurable defect weight system, calculating the overall coil quality score, and automatically outputting the quality grade. This invention achieves accurate detection, quantitative evaluation, and refined quality grading of surface defects in cold-rolled strip steel, effectively improving the intelligence and precision of surface quality control of cold-rolled strip steel, and can be widely applied to the online quality inspection process of various cold-rolled strip steel production enterprises.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an intelligent method for determining the surface quality of cold-rolled strip steel based on image processing. Background Technology

[0002] Currently, machine vision methods are mainly used for surface quality inspection of cold-rolled strip steel. Existing technologies, such as the surface quality inspection system for cold-rolled strip steel developed by the Design and Research Institute Co., Ltd. of Beijing University of Science and Technology, include a hardware configuration of a line scan camera and a light source. It acquires images of the strip steel surface, performs image analysis, identifies surface defects, and displays them on a screen. Errors can be corrected manually.

[0003] However, the existing technology still has the following technical problems: First, the surface of cold-rolled strip steel has the reflective properties of a metallic mirror. Under a single light source, the high-light areas are overexposed and the low-light areas are underexposed, resulting in the loss of local information and making it difficult to effectively collect and identify minute defects.

[0004] Second, the surface of cold-rolled strip steel has periodic rolling textures, which leads to a non-stationary distribution of background grayscale. Conventional global threshold or fixed window threshold segmentation methods are difficult to accurately distinguish between real defects and texture background interference, resulting in a high false detection rate and false negative rate.

[0005] Third, existing technologies can only output defect categories and cannot quantify the severity of defects. Defects of the same type can have significantly different impacts on product quality due to differences in length, width, and depth. Existing technologies cannot distinguish between minor scratches and severe scratches.

[0006] Fourth, existing technologies only output a list of defects, and the quality grade determination of the entire coil of strip steel still relies on comprehensive manual judgment, which is highly subjective, lacks standardized criteria, and cannot achieve automation and standardization of quality judgment.

[0007] Therefore, this invention proposes an intelligent method for determining the surface quality of cold-rolled strip steel based on image processing. Summary of the Invention

[0008] The purpose of this invention is to solve the problems in the prior art, such as information loss due to metal reflection, interference with the segmentation of rolling texture, inability to quantify the severity of defects, and reliance on manual quality judgment. To this end, an intelligent method for judging the surface quality of cold-rolled strip steel based on image processing is proposed.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: an intelligent method for determining the surface quality of cold-rolled strip steel based on image processing, comprising the following steps: Step S1: A line scan camera is used in conjunction with a composite light source that alternates between bright and dark fields to alternately acquire bright field images and dark field images of the steel strip surface. The two images are then fused to eliminate the local information loss caused by the reflection of the metal mirror and obtain a preprocessed image. Step S2: For the background grayscale fluctuation caused by the rolling texture on the surface of cold-rolled strip steel, a local dynamic threshold with the window size dynamically changing with the background texture period is used to segment the preprocessed image, and the real defect area is obtained through morphological screening. Step S3: Extract the geometric features of the real defect area, the texture features based on the gray-level co-occurrence matrix, and the frequency domain features reflecting the repetition period of the defect to construct a multi-dimensional feature vector. Step S4: Identify the defect category based on multidimensional feature vectors, reconstruct the three-dimensional morphology of the defect based on the photometric difference between the bright field image and the dark field image, and fuse the two-dimensional geometric parameters and three-dimensional depth parameters of the defect to calculate the defect severity index. Step S5: Establish a configurable defect weighting system that is linked to the product's intended use. Calculate the overall roll quality score based on defect category weights, defect severity index, and defect density, and automatically output the quality level.

[0010] The beneficial effects of the technical solution provided by this invention include at least the following: This invention employs a linear array camera in conjunction with a composite light source that alternates between bright and dark fields to acquire bright and dark field images of the steel strip surface and then fuses these images. This eliminates the loss of local information caused by metal mirror reflection and improves the ability to acquire minute defects.

[0011] This invention segments preprocessed images by using a local dynamic threshold where the window size changes periodically with the background texture, and obtains the real defect areas through morphological screening. This enables accurate segmentation of the non-stationary distribution of background grayscale caused by the rolling texture of cold-rolled strip steel, reducing the false detection rate and the missed detection rate.

[0012] This invention constructs a multidimensional feature vector by extracting geometric features of real defect areas, texture features based on gray-level co-occurrence matrices, and frequency domain features reflecting the repetition period of defects. This vector can comprehensively characterize the multidimensional attributes of defects and provide a reliable basis for defect classification and quantification.

[0013] This invention identifies defect categories based on multidimensional feature vectors and reconstructs the three-dimensional morphology of defects based on the luminance difference between bright-field and dark-field images. It then fuses the two-dimensional geometric parameters and three-dimensional depth parameters of the defects to calculate a defect severity index, enabling a quantitative assessment of defect severity and distinguishing between minor and severe defects.

[0014] This invention establishes a configurable defect weighting system linked to product application. Based on defect category weight, defect severity index, and defect density, it calculates the overall quality score of the coil and automatically outputs the quality grade. This enables automated and standardized judgment of the overall quality of strip steel, eliminating the subjectivity and inconsistent standards caused by manual judgment. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the method flow provided in an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an image processing-based intelligent method for determining the surface quality of cold-rolled strip steel proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0020] The following description, in conjunction with the accompanying drawings, details the specific scheme of the intelligent surface quality determination method for cold-rolled strip steel based on image processing provided by the present invention.

[0021] Please see Figure 1 The diagram illustrates a flowchart of an intelligent method for determining the surface quality of cold-rolled strip steel based on image processing, according to an embodiment of the present invention, comprising the following steps: Step S1: A line scan camera is used in conjunction with a composite light source that alternates between bright and dark fields to alternately acquire bright field images and dark field images of the steel strip surface. The two images are then fused to eliminate the local information loss caused by the reflection of the metal mirror and obtain a preprocessed image. Step S2: For the background grayscale fluctuation caused by the rolling texture on the surface of cold-rolled strip steel, a local dynamic threshold with the window size dynamically changing with the background texture period is used to segment the preprocessed image, and the real defect area is obtained through morphological screening. Step S3: Extract the geometric features of the real defect area, the texture features based on the gray-level co-occurrence matrix, and the frequency domain features reflecting the repetition period of the defect to construct a multi-dimensional feature vector. Step S4: Identify the defect category based on multidimensional feature vectors, reconstruct the three-dimensional morphology of the defect based on the photometric difference between the bright field image and the dark field image, and fuse the two-dimensional geometric parameters and three-dimensional depth parameters of the defect to calculate the defect severity index. Step S5: Establish a configurable defect weighting system that is linked to the product's intended use. Calculate the overall roll quality score based on defect category weights, defect severity index, and defect density, and automatically output the quality level.

[0022] It should be noted that a line scan camera refers to an industrial camera that uses a linear image sensor to acquire one line of image data at a time, forming a two-dimensional image through the continuous movement of the strip.

[0023] A composite light source with alternating bright and dark fields refers to a lighting system consisting of a bright field light source and a dark field light source. The two light sources are lit alternately in a preset time sequence. The bright field light source illuminates the strip surface at a large angle, while the dark field light source glides across the strip surface at a small angle.

[0024] Bright-field images are images of the steel strip surface acquired under bright-field illumination. These images have a strong specular reflection component and a weak diffuse reflection component.

[0025] Dark-field images refer to images of the steel strip surface acquired under dark-field illumination. These images have strong diffuse reflection components and weak specular reflection components.

[0026] Image fusion refers to the pixel-level weighted merging of bright-field and dark-field images according to a certain coefficient to generate a new image containing both types of illumination information.

[0027] Metallic specular reflection refers to the phenomenon where the smooth surface of cold-rolled steel strip produces a mirror-like directional reflection of incident light, resulting in overexposure or underexposure in local areas of the image.

[0028] The preprocessed image refers to the image obtained by pixel-level weighted fusion of the original field image and the original dark field image. This eliminates the local information loss caused by the reflection of the metal mirror and provides a high-quality image foundation for subsequent defect detection.

[0029] Rolling texture refers to the periodic patterns formed on the surface of strip steel along the rolling direction during the cold rolling process, which is an inherent surface feature of cold-rolled strip steel.

[0030] Non-stationary distribution of background grayscale refers to the periodic fluctuation of background grayscale values ​​on the strip surface along the rolling direction due to the presence of rolling textures, which does not satisfy the statistical stationarity assumption.

[0031] The local dynamic threshold where the window size changes periodically with the background texture means that the size of the segmentation threshold calculation window is not fixed, but dynamically adjusted according to the period length of the rolling texture on the strip surface. The length direction of the window is parallel to the texture direction, and the width direction of the window is perpendicular to the texture direction.

[0032] The true defect area refers to the image area on the strip surface that actually has defects, which is finally determined after a series of image segmentation and filtering operations. It can accurately reflect the actual location, shape and size of the defects.

[0033] Geometric features refer to quantitative parameters that describe the shape and size of the defect area, including area, perimeter, aspect ratio of the circumscribed rectangle, roundness, and relative orientation angle with the rolling grain direction as a reference.

[0034] The gray-level co-occurrence matrix is ​​a second-order statistical method for statistically analyzing the spatial distribution of gray levels in an image. It describes texture features by calculating the probability of occurrence of pixel pairs with specific gray values ​​and specific spatial relationships in an image.

[0035] Texture features refer to four second-order statistics extracted from the gray-level co-occurrence matrix: energy, entropy, moment of inertia, and correlation, which are used to describe the texture attributes of defective regions.

[0036] Frequency domain features refer to the characteristic parameters extracted from the spectrum after performing a two-dimensional Fourier transform on the defect region, including radial energy distribution and angular energy distribution.

[0037] Multidimensional feature vectors are high-dimensional vectors formed by concatenating geometric feature vectors, texture feature vectors, and frequency domain feature vectors in a predetermined order. They integrate the morphological, texture, and frequency features of defects, providing comprehensive feature information for accurate defect classification.

[0038] The photometric difference between bright-field and dark-field images refers to the difference in grayscale value of the same defect point under bright-field and dark-field illumination. This difference is related to the three-dimensional morphology of the defect surface.

[0039] The three-dimensional morphology of a defect refers to the surface shape of the defect region in three-dimensional space, including geometric information such as the depth distribution of the defect and the surface normal vector.

[0040] Two-dimensional geometric parameters refer to the length and width of the defect, where the length is defined as the projected length of the defect along the rolling direction, and the width is defined as the maximum width of the defect perpendicular to the rolling direction.

[0041] The three-dimensional depth parameter refers to the average depth and standard deviation of the defect. The average depth reflects the severity of the defect, while the standard deviation reflects the flatness of the bottom of the defect.

[0042] The defect severity index is a comprehensive quantitative index calculated by integrating the two-dimensional geometric parameters and three-dimensional depth parameters of defects. It is used to characterize the degree of impact of defects on the quality of strip steel.

[0043] The configurable defect weighting system linked to product application refers to the preset of different defect category weight vectors for different product types such as automotive outer panels, automotive inner panels, home appliance panels, and general industrial panels, allowing for dynamic adjustment of the impact of each defect on the quality score according to the product application.

[0044] Defect category weight refers to the weighting coefficient set for each surface defect of cold-rolled strip steel according to the sensitivity of different product applications to various types of defects, reflecting the degree of influence of defects on the quality of strip steel for a specific application.

[0045] Defect density refers to the number of defects on a unit area of ​​strip steel surface, measured in units per square meter.

[0046] The overall quality score of a coil refers to a comprehensive index that represents the surface quality of a whole coil of strip steel by quantitatively calculating the defect category weight, defect severity index, defect density and various weighting factors. It is used to characterize the overall quality level of the whole coil of strip steel.

[0047] Quality grade refers to the grade of strip steel classified according to the overall quality score of the coil, including Grade 1, Grade 2 and Grade 3.

[0048] As one specific implementation method, the intelligent method for judging the surface quality of cold-rolled strip steel provided by the present invention is applied to a 1450 mm cold-rolled continuous annealing production line of a steel enterprise.

[0049] Step S1 is implemented as follows: Four 16K resolution linear CCD cameras are used, two covering the upper surface of the strip and two covering the lower surface. The camera line frequency is set to 80kHz, the strip running speed is 800 meters per minute, and the image resolution is 0.1 mm × 0.1 mm. A high-brightness white LED line light source is used, consisting of two groups: a bright field light source and a dark field light source. The bright field light source illuminates the strip surface at a large angle, while the dark field light source illuminates the strip surface at a small angle. The two groups of light sources are alternately lit according to a time sequence, with the alternation frequency synchronized with the camera line frequency. In each scanning cycle, the bright field light source is lit first to acquire one line of bright field images, and then the dark field light source is lit to acquire one line of dark field images. The acquired bright field images and dark field images are then pixel-level weighted fusion to eliminate local overexposure or underexposure caused by metal specular reflection, resulting in a pre-processed image.

[0050] Specifically, the first set of images is acquired using a preset isochronous alternation method. The gradient direction histograms of the first bright-field image and the first dark-field image are calculated, and the main peak direction is extracted as the rolling texture direction. Based on the angle between the rolling texture direction and the light source illumination direction, the ratio of bright-field acquisition time to dark-field acquisition time, as well as the relative magnitude of the bright-field fusion coefficient and the dark-field fusion coefficient, are determined. The entire original bright-field image and the entire original dark-field image are acquired according to the determined time ratio, and pixel-level weighted fusion is performed according to the determined fusion coefficient to generate a preprocessed image.

[0051] The specific implementation of step S2 is as follows: Based on the determined rolling texture direction, the window shape of the local dynamic threshold is set to a rectangular window that is elongated along this direction. The length of the window is parallel to the texture direction, and the width of the window is perpendicular to the texture direction. The peak spacing of the grayscale projection curve is analyzed along the rolling texture direction, and the texture period length P is statistically obtained. The length L = m × P and the width W = n × P of the rectangular window are set, where m ≥ 2 and n ≤ 1. The grayscale mean μ and grayscale standard deviation σ are calculated in each window, and the dynamic segmentation threshold is calculated according to T = μ + k × σ, where k is dynamically adjusted with the grayscale variance in the local window. Pixels with grayscale values ​​lower than the threshold are marked as defect candidate points, resulting in a binarized defect region. A structuring element that is elongated along the texture direction is used for closing operations, and then a circular structuring element is used for opening operations. Finally, connected component analysis is performed, and an adaptive area filtering threshold is set according to the aspect ratio of the connected components to obtain the true defect region.

[0052] The specific implementation of step S3 is as follows: Calculate the area, perimeter, aspect ratio of the circumscribed rectangle, and roundness of the actual defect region. Calculate the relative direction angle (the difference between the defect direction angle and the rolling texture direction) using the rolling texture direction as a reference. Calculate the ratio of the projected length of the defect along the rolling direction to its length perpendicular to the rolling direction, forming a geometric feature vector. Construct gray-level co-occurrence matrices along the rolling texture direction, perpendicular to the rolling texture direction, and diagonally. Calculate the energy, entropy, moment of inertia, and correlation in each direction. Weight the feature parameters of the three directions according to preset weights to form a texture feature vector. Perform a two-dimensional Fourier transform on the actual defect region, expand the spectrum in polar coordinates, and extract the radial and angular energy distributions to form a frequency domain feature vector. Concatenate the geometric feature vector, texture feature vector, and frequency domain feature vector to construct a multidimensional feature vector.

[0053] The specific implementation of step S4: Input the multi-dimensional feature vector into a lightweight convolutional neural network. This network adopts depthwise separable convolution and channel attention mechanisms, with a parameter size not exceeding 5MB and a single-frame inference time not exceeding 5 milliseconds. The network outputs a defect category probability distribution. Simultaneously, the original bright-field and dark-field images obtained in step S1 are reused to extract the grayscale values ​​of the same point within the defect region under bright-field and dark-field illumination. The photometric stereo method is used to solve for the normal vector, and a specular reflection component model is introduced during the solution: the grayscale value of the dark-field image is used as the specular reflection component, and the specular reflection normal vector is solved using the photometric stereo method; the grayscale value of the bright-field image is subtracted from the grayscale value of the dark-field image to obtain the diffuse reflection component, and the diffuse reflection normal vector is solved using the photometric stereo method; the specular reflection normal vector and the diffuse reflection normal vector are weighted and fused to obtain the final normal vector of the defect surface. The normal vector field is integrally reconstructed to obtain a three-dimensional depth map of the defect region, and the average depth H_est and depth standard deviation H_std of the defect are extracted. Measure the projected length R of the defect along the rolling direction and the maximum width K perpendicular to the rolling direction, and count the number of defects per unit area E. Calculate the defect severity index using the formula S=α×(R / R_0)×(K / K_0)×(H_est / H_0)×(E / E_0)×(1+β×H_std / H_est).

[0054] In this embodiment, one defect was detected, and the network output was a longitudinal scratch. Measurements showed R = 25 mm, K = 0.5 mm, H_est = 8 μm, H_std = 2 μm, and E = 15 scratches per square meter. Taking R_0 = 10 mm, K_0 = 0.2 mm, H_0 = 5 μm, E_0 = 10 scratches per square meter, α = 1, and β = 0.5, the calculated defect severity index S = 20, classifying it as a moderate defect.

[0055] Step S5 is implemented as follows: Different defect category weights are preset for automotive outer panels, automotive inner panels, appliance panels, and ordinary industrial panels. In this embodiment, the strip steel produced is used for appliance panels, and the preset weight for scratch defects is q_i = 1.2. The strip steel is divided into an edge region and a middle region along its width, with the edge region being 100 mm wide. An edge defect weighting factor F_edge is defined: F_edge = 1.5 when the defect is located in the edge region and F_edge = 1.0 when the defect is located in the middle region. The defect clustering effect factor F_cluster is calculated; in this embodiment, the defect distribution is relatively uniform, so F_cluster = 1.0. The length of the entire strip steel coil is 2000 meters, the defect density D_ensity = 15 defects per square meter, and the reference density D_0 = 10 defects per square meter. The overall quality score of the coil is calculated using the formula Q = [Σ(q_i × S_i) + γ × (D_ensity / D_0)] × F_cluster × F_edge, where γ = 5. The calculated value is Q = 464.3. The preset first scoring threshold T_1 = 300, and the second scoring threshold T_2 = 600. Since 464.3 is greater than 300 but less than 600, the strip steel is determined to be a Grade II product, a downgrade instruction is output, and the defect type and location are marked on the steel coil label. For strip steel with a score that reaches or exceeds the second scoring threshold, it is determined to be a Grade III product, a scrap or rework instruction is output, and an alarm is triggered.

[0056] This embodiment has been running for three months, with a defect detection rate of 97.2%, a defect missed detection rate of 0.08%, a defect false detection rate of 0.35%, a classification accuracy rate of 94.6%, an average processing time of 3.8 milliseconds per frame, and a continuous operating stability of 99.95%.

[0057] Step S1 further includes the following sub-steps: S1-1, using a preset isochronous alternation method, controls the line scan camera to acquire the first set of images in conjunction with bright field light source and dark field light source, forming the first bright field image and the first dark field image respectively; S1-2, Calculate the gradient direction histogram of the first bright field image and the first dark field image, and extract the main peak direction as the rolling texture direction; S1-3, Determine the ratio of bright field acquisition time to dark field acquisition time based on the angle between the rolling texture direction and the light source illumination direction; S1-4, according to a determined time ratio, control the linear array camera to collect images of the strip surface in conjunction with bright field light source and dark field light source. In each scanning cycle, first collect the original bright field image according to the bright field acquisition time, and then collect the original dark field image according to the dark field acquisition time. The process is repeated alternately to form the original bright field image and the original dark field image respectively. S1-5, calculate the gray-level variance of each pixel in the preset neighborhood window of the whole bright field original image and the whole dark field original image respectively, and obtain the bright field variance map and the dark field variance map. S1-6, Determine the relative magnitudes of the bright-field blending coefficient and the dark-field blending coefficient based on the angle between the rolling texture direction and the light source illumination direction; S1-7. According to the determined fusion coefficient, the original bright field image and the original dark field image are fused at the pixel level to generate a preprocessed image. The fusion formula is: I_fusion=w×I_bright+(1-w)×I_dark, where I_fusion is the preprocessed image, I_bright is the original bright field image, I_dark is the original dark field image, and w is the fusion coefficient of the bright field image.

[0058] Furthermore, in sub-steps S1-3, the ratio of bright-field acquisition time to dark-field acquisition time is determined according to the following formula: When the angle between the rolling texture direction and the light source illumination direction is less than the preset angle threshold, the bright field acquisition time: dark field acquisition time = tan(angle): 1; When the included angle is equal to the preset included angle threshold, the ratio of bright field acquisition time to dark field acquisition time is 1:1. When the included angle is greater than the preset included angle threshold, the bright field acquisition time : dark field acquisition time = 1 : tan(90° - included angle).

[0059] Furthermore, in sub-steps S1-6, the relative magnitudes of the bright-field blending coefficient and the dark-field blending coefficient are determined according to the following formula: When the angle between the rolling texture direction and the light source illumination direction is less than the preset angle threshold, the bright field fusion coefficient = 1 / (1+tan(angle)). When the included angle is equal to the preset included angle threshold, the bright field fusion coefficient is 0.5; When the included angle is greater than the preset included angle threshold, the bright field fusion coefficient = tan(included angle) / (1 + tan(included angle)).

[0060] It should be noted that the isochronous alternation method refers to controlling the bright field light source and the dark field light source to be turned on alternately at equal time intervals, in conjunction with the line scan camera to complete image acquisition, providing a fixed timing reference for the initial image acquisition.

[0061] A gradient orientation histogram is a histogram formed by statistically analyzing the gradient direction of each pixel in an image and organizing it into directional intervals. It can intuitively reflect the main extension direction of the texture in the image.

[0062] The direction of light source illumination refers to the direction in which the light from the light source and the dark light source is projected onto the surface of the strip steel. It is the reference direction for calculating the angle with the rolling texture direction.

[0063] Brightfield acquisition time refers to the length of time during which the brightfield light source is turned on and the line scan camera acquires brightfield images within a single scanning cycle.

[0064] Dark field acquisition time refers to the length of time during which the dark field light source is turned on and the line scan camera acquires dark field images within a single scanning cycle.

[0065] The scanning cycle refers to the complete time period for a line scan camera to complete one bright-field image acquisition and one dark-field image acquisition in conjunction with bright-field and dark-field light sources.

[0066] Bright-field raw images refer to images of the strip surface acquired by a line scan camera under bright-field light source illumination without any processing, which can clearly show the diffuse reflection characteristics of the strip surface.

[0067] Dark field raw images refer to unprocessed images of the strip surface captured by a line scan camera under single illumination from a dark field light source. They clearly show the specular reflection characteristics and defect edges of the strip surface.

[0068] Gray-level variance refers to the degree of dispersion of gray-level values ​​of all pixels within a certain neighborhood window of an image, and can reflect the fluctuation and change of gray-level in that area.

[0069] A brightfield variance map is an image formed by using the neighborhood grayscale variance of each pixel in the original brightfield image as the pixel value. It can reflect the grayscale uniformity of each region of the brightfield image.

[0070] A dark field variance map is an image formed by using the neighborhood gray-level variance of each pixel in the original dark field image as the pixel value. It can reflect the gray-level uniformity of each region of the dark field image.

[0071] Bright-field fusion coefficients refer to the weighting coefficients set for the original bright-field image during the pixel-level weighted fusion process, which determine the information proportion of the bright-field image in the preprocessed image.

[0072] The dark field fusion coefficient refers to the weighting coefficient set for the original dark field image during the pixel-level weighted fusion process, which determines the information proportion of the dark field image in the preprocessed image.

[0073] Pixel-level weighted fusion refers to an image fusion method that multiplies the gray values ​​of the same pixel location in the original bright field image and the original dark field image by the corresponding fusion coefficients and then adds them together to generate a new pixel value. It can achieve fine-grained information fusion between two images.

[0074] The preset angle threshold is a critical value set for adjusting the proportion of light source acquisition time and image fusion coefficient. It is the criterion for dividing the angle range and executing different calculation formulas.

[0075] In one specific embodiment, the intelligent method for determining the surface quality of cold-rolled strip steel provided by the present invention is applied to a 1450 mm cold-rolled continuous annealing production line of a steel enterprise. The production line rolls strip steel with a width ranging from 700 mm to 1450 mm and a rolling speed ranging from 30 m / min to 120 m / min.

[0076] In the image acquisition stage, four 16K resolution linear CCD cameras were used, two covering the upper surface of the strip and two covering the lower surface. The camera line frequency was set to 80kHz, the strip speed was 800 meters per minute, and the image resolution was 0.1 mm × 0.1 mm. High-brightness white LED line light sources were used, consisting of two groups: a bright field light source and a dark field light source. The bright field light source illuminated the strip surface at a 45-degree angle, while the dark field light source swept across the strip surface at a 15-degree angle.

[0077] First, using a preset isochronous alternation mode, the linear scan camera, in conjunction with bright-field and dark-field light sources, acquires the first set of images. In the isochronous alternation mode, the bright-field illumination time and the dark-field illumination time each occupy half of the scanning cycle, each being 6.25 microseconds. The acquired first bright-field image and first dark-field image are both obtained, with a resolution of 16K pixels multiplied by 5000 lines.

[0078] Calculate the gradient direction histograms of the first bright-field image and the first dark-field image. For each pixel, calculate the horizontal and vertical gradients to obtain the gradient direction and magnitude. Divide the directional range from 0 to 180 degrees into 9 intervals of 20 degrees each, and sum the gradient magnitudes within each interval to form a gradient direction histogram. Extract the direction corresponding to the interval with the largest magnitude in the histogram as the main peak direction; this direction is the rolling texture direction. In this embodiment, the rolling texture direction is horizontal, consistent with the strip running direction.

[0079] The ratio of bright-field acquisition time to dark-field acquisition time is determined based on the angle between the rolling texture direction and the light source illumination direction. In this embodiment, the angle between the bright-field light source illumination direction and the normal to the strip surface is 45 degrees, the angle between the dark-field light source illumination direction and the normal to the strip surface is 75 degrees, and the rolling texture direction is a horizontal direction of 0 degrees. The angle between the rolling texture direction and the light source illumination direction is 45 degrees. A preset angle threshold is 45 degrees. When the angle equals the preset angle threshold, the ratio of bright-field acquisition time to dark-field acquisition time is 1:1. Therefore, the bright-field acquisition time is set to 6.25 microseconds, and the dark-field acquisition time is set to 6.25 microseconds.

[0080] According to a predetermined time ratio, the linear scan camera, in conjunction with bright-field and dark-field light sources, acquires images of the strip surface. Within each scanning cycle, a line of bright-field raw images is acquired first at a time of 6.25 microseconds, followed by a line of dark-field raw images at a time of 6.25 microseconds, alternating between the two. As the strip moves continuously, the linear scan camera acquires images continuously, forming both full-frame bright-field and full-frame dark-field raw images, each with a resolution of 16K pixels multiplied by 20,000 lines.

[0081] Calculate the grayscale variance within a preset neighborhood window for each pixel in both the original bright-field and original dark-field images to obtain the bright-field variance map and dark-field variance map. The preset neighborhood window size is 5 pixels by 5 pixels. For each pixel, calculate the mean grayscale value of its 25 pixels within a 5×5 neighborhood, then calculate the sum of the squares of the differences between each pixel's grayscale value and the mean, and divide by 25 to obtain the grayscale variance. This process is repeated for all pixels in the entire image to obtain the bright-field variance map and dark-field variance map with the same resolution as the original image.

[0082] The relative magnitudes of the bright-field blending coefficient and the dark-field blending coefficient are determined based on the angle between the rolling texture direction and the light source illumination direction. In this embodiment, the angle is 45 degrees, which is equal to a preset angle threshold. The bright-field blending coefficient w is set to 0.5, and the dark-field blending coefficient 1-w is set to 0.5.

[0083] According to the determined fusion coefficients, the original bright-field image and the original dark-field image are fused pixel-wise with weighted averages. For each pixel location in the image, the grayscale value I_bright of that location in the original bright-field image and the grayscale value I_dark of that location in the original dark-field image are taken, and the fused grayscale value is calculated using the fusion formula I_fusion = 0.5 × I_bright + 0.5 × I_dark. This process is repeated for all pixel locations in the entire image to generate a preprocessed image. In the fused preprocessed image, the pixel values ​​of overexposed areas with specular reflection in the bright-field image are compensated by the normal pixel values ​​of the corresponding areas in the dark-field image, and the pixel values ​​of underexposed areas in the dark-field image are compensated by the normal pixel values ​​of the corresponding areas in the bright-field image, thus eliminating the local information loss caused by metallic specular reflection.

[0084] Step S2 further includes the following sub-steps: S2-1, Based on the determined rolling texture direction, the window shape of the local dynamic threshold is set to a rectangular window that is stretched along this direction, with the window length direction parallel to the texture direction and the window width direction perpendicular to the texture direction; S2-2, Based on the period length P of the rolling texture, set the length L of the rectangular window to m×P and the width W to n×P, where m≥2 and n≤1; S2-3, within the defined rectangular window, calculate the dynamic segmentation threshold, specifically using the formula T(x,y) = μ(x,y). (x, y), where μ(x, y) is the mean gray value within the window, σ(x, y) is the standard deviation of gray values ​​within the window, and k is an adaptive coefficient that dynamically adjusts with the variance of gray values ​​within the local window; S2-4: Mark pixels with gray values ​​lower than the dynamic segmentation threshold T(x,y) as candidate defect points to obtain the binarized defect region; S2-5, Morphological processing is performed on the binarized defect area. First, a structuring element that is elongated along the texture direction is used to perform a closing operation, and then a circular structuring element is used to perform an opening operation. S2-6. Perform connected component analysis on the processed binarized image, and set an adaptive area filtering threshold based on the aspect ratio of the connected components to obtain the real defect area.

[0085] It should be noted that the rolling texture direction refers to the extension direction of the regular texture formed on the surface of cold-rolled strip during the rolling process. It is the inherent texture direction of the strip surface and provides a reference for setting the shape and direction of the local dynamic threshold window.

[0086] Local dynamic threshold refers to the image segmentation threshold dynamically calculated based on the grayscale characteristics of local areas on the surface of the strip steel. It is not a fixed value and can adapt to the grayscale fluctuations in different areas of the surface of cold-rolled strip steel.

[0087] A rectangular window is a rectangular analysis area defined for calculating the local dynamic threshold, and it is the basic unit for extracting local image grayscale information of the strip surface.

[0088] The period length of the rolling texture refers to the distance between two adjacent texture units in the regular rolling texture on the surface of cold-rolled strip steel, and it is the core reference parameter for determining the size of the rectangular window.

[0089] The dynamic segmentation threshold is a critical gray value calculated based on the mean and standard deviation of gray values ​​within a rectangular window, used to distinguish between defective pixels on the strip surface and normal base material pixels.

[0090] The mean gray level within a window refers to the arithmetic mean of the gray levels of all pixels within a defined rectangular window, reflecting the overall gray level of the image within the window.

[0091] The standard deviation of grayscale within a window refers to the degree of dispersion of the grayscale values ​​of all pixels within a defined rectangular window relative to the grayscale mean, reflecting the grayscale fluctuations of the image within the window.

[0092] The adaptive coefficient is a coefficient that is dynamically adjusted based on the gray-level variance within a local window. It is used to balance the weights of the gray-level mean and gray-level standard deviation in the calculation of the dynamic segmentation threshold.

[0093] Defect candidate points refer to pixels with gray values ​​lower than the dynamic segmentation threshold. These are initially identified as suspected defective pixels and require further screening to eliminate false defect points.

[0094] Binarized defect region refers to the black and white image region formed by marking defect candidate points as the first value and normal pixels as the second value, which can intuitively present the distribution and shape of suspected defects.

[0095] Morphological processing refers to image operations performed on binary defect regions based on mathematical morphology theory, used to optimize the contour morphology of defect regions and eliminate false defects and defect contour breaks.

[0096] Structural elements refer to the basic geometric units used in morphological processing to perform calculations on binary defect regions. Their shape and orientation are determined based on the characteristics of strip rolling texture.

[0097] Closing operation refers to a morphological operation that first performs dilation operation on the binarized defect region and then performs erosion operation. It is used to connect the fracture contour of the defect region and fill the tiny holes inside the defect.

[0098] Opening operation refers to a morphological operation that first performs erosion operation on the binarized defect region and then performs dilation operation. It is used to eliminate tiny pseudo-defect points around the defect region and smooth the defect contour.

[0099] Connected component analysis refers to region analysis performed on a binarized image after morphological processing. It divides adjacent pixels with the same numerical value into a connected component, and each connected component represents an independent suspected defect region.

[0100] The aspect ratio refers to the ratio of the length to the width in the smallest bounding rectangle of a connected region. It can reflect the morphological characteristics of suspected defective areas and provide a basis for screening actual defects.

[0101] The adaptive area filtering threshold refers to the area critical value dynamically set according to the aspect ratio of the connected components. Different aspect ratios of connected components correspond to different area filtering standards, which are used to eliminate pseudo-defect connected components with excessively small areas.

[0102] In one specific embodiment, the intelligent method for judging the surface quality of cold-rolled strip steel provided by the present invention is applied to a 1450 mm cold-rolled continuous annealing production line of a steel enterprise. The production line rolls strip steel with a width range of 700 mm to 1450 mm, a rolling speed of 30 m / min to 120 m / min, and the detection range of the period length P of the rolling texture on the surface of the strip steel is 0.2 mm to 1.0 mm.

[0103] At the strip surface inspection station on this production line, the rolling texture direction of the cold-rolled strip to be inspected has been determined through preliminary steps to be the length extension direction of the strip. Based on this rolling texture direction, the window shape of the local dynamic threshold is set to a rectangular window elongated along the length direction of the strip. The length direction of the window is parallel to the length direction of the strip, and the width direction of the window is parallel to the width direction of the strip, i.e., perpendicular to the rolling texture direction. For this batch of strip to be inspected, image analysis determined that the period length P of its rolling texture is 0.5 mm. Accordingly, the length L of the rectangular window is set to 3 × P, i.e., 1.5 mm, and the width W is set to 0.8 × P, i.e., 0.4 mm, to match the texture characteristics of this batch of strip.

[0104] Within the defined rectangular window, the dynamic segmentation threshold is calculated window by window, according to the formula T(x,y)=μ(x,y). The calculation is performed on (x,y), where μ(x,y) is the mean gray value of all pixels in the current rectangular window, σ(x,y) is the standard deviation of gray values ​​of all pixels in the current rectangular window, and the adaptive coefficient k is dynamically adjusted according to the gray value variance in the local window. When the gray value variance is in the range of 0 to 50, k is 1.0; when the gray value variance is in the range of 50 to 100, k is 1.5; and when the gray value variance is above 100, k is 2.0. The threshold calculation weights are used to balance different gray value fluctuation areas.

[0105] The gray value of each pixel in the surface image of the strip to be inspected is compared with the dynamic segmentation threshold T(x,y) calculated by the corresponding rectangular window. Pixels with gray values ​​lower than the dynamic segmentation threshold are uniformly marked as defect candidate points, and pixels with gray values ​​higher than or equal to the dynamic segmentation threshold are marked as normal base material pixels, forming a binary defect region composed of black and white.

[0106] Morphological processing was performed on the binarized defect region. First, a rectangular structural element elongated along the length of the strip, i.e. the rolling texture direction, was used for closing operations. The length of this structural element was 0.6 mm and the width was 0.1 mm, which completed the broken connection of the defect contour and the filling of the internal micro-holes. Then, a circular structural element with a diameter of 0.2 mm was used for opening operations to eliminate the micro-pseudo-defect points around the defect region and smooth the defect contour.

[0107] Connectivity analysis was performed on the binarized image after morphological processing. Adjacent defect candidate pixels were divided into independent connected components. The minimum bounding rectangle of each connected component was calculated to obtain the aspect ratio of each component. An adaptive area filtering threshold was set according to the aspect ratio: the area filtering threshold was set to 20 pixels for connected components with an aspect ratio of 1 to 3, 10 pixels for connected components with an aspect ratio of 3 to 10, and 5 pixels for connected components with an aspect ratio of more than 10. Pseudo-defect connected components with an area lower than the corresponding adaptive area filtering threshold were removed, while connected components with an area higher than the corresponding threshold were retained. Finally, the true defect area on the surface of this batch of cold-rolled strip steel was obtained, providing an accurate image basis for subsequent defect feature extraction and quality judgment.

[0108] Step S3 further includes the following sub-steps: S3-1 Calculate the area, perimeter, aspect ratio of the circumscribed rectangle, and roundness of the actual defect region. Calculate the relative direction angle with the rolling texture direction as a reference, and the ratio of the length of the defect along the rolling direction to the length perpendicular to the rolling direction to form a geometric feature vector. The relative direction angle is the difference between the defect direction angle and the rolling texture direction. S3-2, construct gray-level co-occurrence matrices along the rolling texture direction, perpendicular to the rolling texture direction, and diagonal direction respectively, calculate the energy, entropy, moment of inertia, and correlation of each direction, and weight and fuse the feature parameters of the three directions according to preset weights to form a texture feature vector; S3-3, Perform a two-dimensional Fourier transform on the real defect area, expand the spectrum in polar coordinates, extract the radial energy distribution and angular energy distribution, and form a frequency domain feature vector; S3-4 concatenates the geometric feature vector, texture feature vector, and frequency domain feature vector to construct a multidimensional feature vector.

[0109] It should be noted that a geometric feature vector is a vector formed by arranging multiple geometric morphological feature parameters of a real defect area in a predetermined order. It is a digital carrier that represents the spatial morphological features of a defect and provides basic morphological information for defect identification.

[0110] The aspect ratio of the circumscribed rectangle refers to the ratio of the length of the longer side to the length of the shorter side of the smallest circumscribed rectangle of the actual defect area. It is a geometric parameter that reflects the overall tensile characteristics of the defect.

[0111] Circularity refers to a geometric parameter that characterizes how close the outline of a real defect area is to a circle. The larger the value, the closer the defect outline is to a circle, and it can distinguish between circular defects and strip-shaped defects.

[0112] The defect orientation angle is the angle between the main extension direction of the actual defect area and the preset horizontal direction. It is a geometric parameter that reflects the overall orientation of the defect.

[0113] The relative orientation angle refers to the difference between the defect orientation angle and the rolling texture direction, with the rolling texture direction as the reference. It can characterize the relative relationship between the defect orientation and the inherent texture direction of the strip.

[0114] Energy refers to the sum of squares of all elements in the gray-level co-occurrence matrix. It is a texture parameter that reflects the uniformity and coarseness of the surface texture of defects. The higher the energy, the more uniform and delicate the texture.

[0115] Entropy refers to the information entropy calculated based on the gray-level co-occurrence matrix. It is a texture parameter that reflects the complexity of the texture on a defective surface. The larger the entropy value, the more complex and irregular the texture.

[0116] Moment of inertia refers to the sum of squared differences of elements along the main diagonal in a gray-level co-occurrence matrix. It is a texture parameter that reflects the contrast and sharpness of the texture on a defective surface. The larger the moment of inertia, the higher the texture contrast.

[0117] Correlation refers to the correlation coefficient calculated based on the gray-level co-occurrence matrix. It is a texture parameter that reflects the correlation of gray values ​​on the defect surface in the row or column direction. The greater the correlation, the more regular the texture direction.

[0118] A texture feature vector is a vector formed by arranging multiple texture feature parameters in a predetermined order. It is a digital carrier that characterizes the texture features of a defective surface and can distinguish defects that are similar in shape but different in texture.

[0119] Two-dimensional Fourier transform is a mathematical transformation that converts a spatial domain image of a real defect region into a frequency domain spectrum, enabling the analysis of the spatial characteristics of defects by converting them into frequency characteristics.

[0120] A spectrum diagram is a frequency domain image obtained by performing a two-dimensional Fourier transform on a real defect region. It contains spatial frequency distribution information of the defect and can reflect the periodicity and regularity of the defect.

[0121] Radial energy distribution refers to the energy distribution characteristics along the radial direction after the spectrum is expanded in polar coordinates. It can reflect the energy intensity of defects at different spatial frequencies and characterize the overall texture of defects.

[0122] Angular energy distribution refers to the energy distribution characteristics along the angular direction after the spectrum is expanded in polar coordinates. It can reflect the frequency energy strength of defects in different directions and characterize the directional regularity of defects.

[0123] Frequency domain feature vectors are vectors formed by arranging the feature parameters of radial and angular energy distributions in a predetermined order. They are digital carriers that characterize the frequency domain features of defects and can effectively identify periodic defects.

[0124] In one specific embodiment, the intelligent surface quality judgment method for cold-rolled strip steel provided by the present invention is applied to a 1450 mm cold-rolled continuous annealing production line of a steel enterprise. The production line rolls strip steel with a width range of 700 mm to 1450 mm and a rolling speed of 30 m / min to 120 m / min. The matching surface inspection system is an 8192 pixel line array camera with a pixel resolution of 0.1 mm per pixel. The actual defect area on the surface of the strip steel has been obtained through the previous steps, including typical cold-rolled strip steel defects such as scratches, pits, and roll marks. The rolling texture direction is the length extension direction of the strip steel.

[0125] For the extracted real defect areas, step S3-1 is first executed. The area of ​​each defect area (i.e., the total number of pixels contained in the defect) and the perimeter (i.e., the number of pixels in the defect outline) are calculated by statistically analyzing the image pixels. The minimum bounding rectangle of the defect is fitted and the aspect ratio is calculated. The roundness is calculated by the contour curvature. The strip length direction is used as the rolling texture reference. The difference between the defect direction angle and the reference is calculated to obtain the relative direction angle. At the same time, the extension length of the defect along the strip length direction and the width perpendicular to the length direction are measured and the ratio of the two is calculated. The six parameters, namely area, perimeter, aspect ratio of the bounding rectangle, roundness, relative direction angle, and the ratio of the length in the rolling direction to the length in the vertical direction, are arranged in order to construct a 6-dimensional geometric feature vector.

[0126] Next, execute step S3-2, setting the step size of the gray-level co-occurrence matrix to 1 pixel. Construct gray-level co-occurrence matrices for each real defect region along the strip length direction (rolling texture direction), the strip width direction (perpendicular to the rolling texture direction), and the 45-degree diagonal direction of the length and width. Calculate the four texture parameters (energy, entropy, moment of inertia, and correlation) for the three directional matrices. Set the weights for the rolling texture direction, perpendicular to the rolling texture direction, and diagonal direction to 0.2. Weight the same texture parameters according to their corresponding weights and fuse them to obtain the fused energy, entropy, moment of inertia, and correlation parameters. Arrange them in order to construct a 4-dimensional texture feature vector.

[0127] Then, step S3-3 is executed. Each real defect region is zero-filled to a fixed pixel size and then subjected to a two-dimensional Fourier transform to obtain the corresponding frequency domain spectrum. The rectangular coordinates of the spectrum are converted to polar coordinates. The angle range of the polar coordinates is 0 to 360 degrees, and the radius range is 0 to the maximum radius of the spectrum. The spectrum is divided into blocks with an angle of 10 degrees and a radius of 2 pixels. The energy value in each block is calculated. The energy change trend parameters of different radii in the radial direction and the peak parameters of energy distribution at different angles in the angular direction are extracted. A total of 8 frequency domain feature parameters are extracted and arranged in order to construct an 8-dimensional frequency domain feature vector.

[0128] Finally, step S3-4 is executed. The 6D geometric feature vector, 4D texture feature vector, and 8D frequency domain feature vector are concatenated in sequence according to the order of geometry, texture, and frequency domain to form an 18D multi-dimensional feature vector. This vector integrates the morphological, texture, and frequency multi-dimensional features of the defect and is input into the lightweight convolutional neural network supporting the production line, providing comprehensive and rich feature inputs for the accurate classification of surface defects of cold-rolled strip steel. In the practical application of this production line, the discrimination accuracy of similar defects such as scratches, pitting, and roll marks is increased to more than 98%, meeting the accuracy requirements of high-speed detection on the production line.

[0129] Among them, in step S4, the following sub-steps are further included: S4-1, input the multi-dimensional feature vector into the lightweight convolutional neural network to output the probability distribution of defect categories. The lightweight convolutional neural network adopts depthwise separable convolution and channel attention mechanism; S4-2, use the entire bright-field original image and the entire dark-field original image to extract the gray values of the same point in the defect area under bright-field illumination and dark-field illumination; S4-3, use the photometric stereo method to solve the normal vector, and introduce a specular reflection component model during the solution. Take the bright-field image as the diffuse reflection component and the dark-field image as the specular reflection component, and solve the normal vector after separating the diffuse reflection component and the specular reflection component; S4-4, perform integral reconstruction on the obtained normal vector field to obtain the three-dimensional depth map of the defect area, and extract the average depth H_est and depth standard deviation H_std of the defect; S4-5, measure the projection length of the defect along the rolling direction and the maximum width perpendicular to the rolling direction, and count the number of defects per unit area; S4-6, calculate the defect severity index, and the specific formula is: S = α×(R / R_0)×(K / K_0)×(H_est / H_0)×(E / E_0)×(1 + β×H_std / H_est), where R is the projection length of the defect along the rolling direction, R_0 is the reference value of the projection length, K is the maximum width of the defect perpendicular to the rolling direction, K_0 is the reference value of the maximum width, H_est is the average depth of the defect, H_0 is the depth reference value, E is the number of defects per unit area, E_0 is the number reference value, H_std is the depth standard deviation of the defect, α is the normalization coefficient, and β is the depth uniformity adjustment coefficient.

[0130] Furthermore, in sub-step S4-3, the steps of separating the diffuse reflection component and the specular reflection component and then solving the normal vector include: Take the gray value of the dark-field image as the specular reflection component and solve the specular reflection normal vector through the photometric stereo method; Subtract the gray value of the dark-field image from the gray value of the bright-field image to obtain the diffuse reflection component; Based on the diffuse reflection component, the diffuse reflection normal vector is solved using the photometric stereo method. The specular reflection normal vector and the diffuse reflection normal vector are weighted and fused to obtain the final normal vector of the defective surface.

[0131] It should be noted that lightweight convolutional neural networks refer to convolutional neural networks that reduce the number of parameters and computational load by simplifying the network structure, while maintaining high feature recognition capabilities, thus meeting the computing power requirements of real-time detection in industrial production lines.

[0132] Depthwise separable convolution refers to a convolution method that decomposes standard convolution into depthwise convolution and pointwise convolution, which can significantly reduce the computational cost and number of parameters in the network and improve the inference speed of the model.

[0133] Channel attention mechanism refers to a mechanism that adaptively weights the importance of different feature channels in a neural network, which can strengthen the information of key feature channels and improve the accuracy of feature recognition.

[0134] The defect category probability distribution refers to the set of probability values ​​of various surface defects of cold-rolled strip steel output by a lightweight convolutional neural network after recognizing the input multidimensional feature vector.

[0135] A bright-field raw image refers to an unprocessed image of the strip surface acquired using bright-field illumination, which clearly presents the diffuse reflection characteristics of the strip surface.

[0136] Dark field raw images refer to unprocessed images of the strip surface acquired using dark field illumination, which clearly present the specular reflection characteristics of the strip surface.

[0137] Photometric stereo method refers to an optical measurement method that uses image grayscale information under different lighting directions to solve for the surface normal vector of an object, and then reconstructs the three-dimensional shape of the object.

[0138] The specular reflection component model is a mathematical model used to separate the specular reflection and diffuse reflection components in the reflected light from the surface of an object, which can improve the accuracy of solving the normal vector of a metal surface.

[0139] The diffuse reflection component refers to the light signal component that is uniformly reflected in all directions from the surface of an object. It can reflect the inherent grayscale characteristics and three-dimensional shape information of the object's surface.

[0140] Specular reflection component refers to the light signal component reflected from the surface of an object along a specific direction. It is easily affected by reflections from metal surfaces and is a major factor interfering with the reconstruction of three-dimensional shapes.

[0141] A normal vector is a vector perpendicular to a point on the surface of an object. It is a core parameter that characterizes the three-dimensional shape of the object's surface and reflects the direction and degree of the surface's tilt.

[0142] A normal vector field is a set of vectors consisting of the normal vectors corresponding to all pixels within a defect region, which can completely characterize the surface three-dimensional morphological features of the defect region.

[0143] Integral reconstruction refers to a mathematical method that calculates the surface normal vector field by integration, transforming the surface normal vector information into surface depth information. It is a key step in realizing three-dimensional topography reconstruction.

[0144] A 3D depth map is an image obtained through integral reconstruction that represents the depth value of each pixel in a defect region. It can intuitively present the 3D protrusion or depression shape of the defect.

[0145] Average depth refers to the arithmetic mean of the depth values ​​of all pixels within the defect area, and is a core parameter characterizing the overall degree of concavity or convexity of the defect.

[0146] Depth standard deviation refers to the degree of dispersion of the depth values ​​of all pixels within a defect region relative to the average depth. It is a parameter characterizing the uniformity of depth on the defect surface.

[0147] Projected length refers to the two-dimensional projection dimension of the defect in the rolling direction, and is a geometric parameter characterizing the extent to which the defect extends along the strip movement direction.

[0148] Maximum width refers to the maximum two-dimensional dimension of the defect perpendicular to the rolling direction, and is a geometric parameter characterizing the degree of lateral expansion of the defect.

[0149] The number of defects per unit area refers to the number of defects appearing on the surface of a strip steel per unit area, and is a parameter characterizing the density of defect distribution.

[0150] The normalization coefficient is a coefficient used to normalize the calculated result of the defect severity index to a fixed numerical range, so that the severity indices of different types of defects are comparable.

[0151] The depth uniformity adjustment coefficient is a coefficient used to adjust the weight of the depth standard deviation in the calculation of the defect severity index, which can highlight the influence of the surface smoothness of the defect on the severity.

[0152] The specular reflection normal vector is the normal vector obtained by solving the specular reflection component using the photometric solid method, which reflects the surface orientation characteristics of the reflective area on the metal surface.

[0153] The diffuse reflection normal vector is a normal vector obtained by solving the diffuse reflection component using the photometric stereo method, which reflects the true three-dimensional morphological characteristics of the defect area.

[0154] In one specific embodiment, the intelligent surface quality judgment method for cold-rolled strip steel provided by the present invention is applied to a 1450 mm cold-rolled continuous annealing production line of a steel enterprise. The production line rolls strip steel with a width ranging from 700 mm to 1450 mm and a rolling speed ranging from 30 m / min to 120 m / min. It is equipped with a surface inspection system with an 8192 pixel line array camera and a pixel resolution of 0.1 mm per pixel. The preset surface defect categories of cold-rolled strip steel include five categories: scratches, pitting, roll marks, scabs, and inclusions. Through the previous steps, an 18-dimensional multi-dimensional feature vector characterizing the defect features and the corresponding bright field and dark field original images have been obtained.

[0155] To address the inspection requirements of this production line, step S4-1 was first executed. A lightweight convolutional neural network using depthwise separable convolution and channel attention mechanisms was built. The convolutional layers of the network were replaced with depthwise separable convolutions, and a channel attention module was added to the network feature fusion layer. After training on labeled samples of five types of defects, an 18-dimensional multi-dimensional feature vector was input into the network. The network's single inference time was controlled within 5 milliseconds. The network output the probability values ​​of the five types of defects corresponding to the feature vector: scratches, pitting, roller marks, scars, and inclusions, forming a defect category probability distribution. The category with the highest probability value was taken as the defect identification result.

[0156] Next, step S4-2 is executed. Based on the pixel coordinates of the real defect area obtained in the previous step, the gray value of the same pixel coordinate point is accurately extracted from the whole bright field original image and the whole dark field original image to form a set of corresponding bright field and dark field gray values ​​of the defect area, ensuring that the gray values ​​extracted from the two images are optical information of the same physical location on the strip surface.

[0157] Then, step S4-3 is executed, introducing the specular reflection component model. The gray values ​​extracted from the original dark field image are used as the specular reflection component and input into the photometric stereo method solution model to obtain the specular reflection normal vector. The gray values ​​of the original bright field image are subtracted from the corresponding gray values ​​of the original dark field image to obtain the diffuse reflection component, and the diffuse reflection normal vector is obtained in the same way. The weight of the diffuse reflection normal vector is set to 0.8 and the weight of the specular reflection normal vector is set to 0.2. The two types of normal vectors are weighted and fused according to this weight to obtain the final normal vector of the defect surface, effectively eliminating the interference of specular reflection on the normal vector solution of the strip metal surface.

[0158] Then, step S4-4 is executed, and the obtained normal vector field is integrated and reconstructed using the Poisson integral method to generate a three-dimensional depth map of the defect area. The depth detection accuracy can reach 0.01 mm. The depth values ​​of all pixels in the defect area are statistically analyzed in the three-dimensional depth map, and the average depth H_est and depth standard deviation H_std are calculated to fully characterize the three-dimensional protrusion or depression features of the defect.

[0159] Then execute step S4-5 to measure the projected length R of the defect along the strip length direction, i.e. the rolling direction, in the two-dimensional defect image, and measure the maximum width K of the defect along the strip width direction, i.e. the direction perpendicular to the rolling direction. Using each square meter of strip surface as the statistical unit, the number of defects per unit area E is obtained.

[0160] Finally, step S4-6 is executed. Based on the production line's quality standards, the preset reference values ​​for projected length R_0 are 10 mm, maximum width K_0 is 5 mm, depth H_0 is 0.1 mm, and quantity E_0 is 5 units per square meter. The normalization coefficient α is set to 0.2, and the depth uniformity adjustment coefficient β is set to 0.3. Substituting the above parameters into the formula S=α×(R / R_0)×(K / K_0)×(H_est / H_0)×(E / E_0)×(1+β×H_std / H_est), the defect severity index S is calculated. The index value is controlled between 0 and 10. The larger the value, the higher the severity of the defect. In the actual application of this production line, this calculation method realizes the quantitative differentiation of different types and degrees of defects, providing an accurate single-defect evaluation basis for the subsequent quality scoring of the entire coil of strip steel.

[0161] Step S5 further includes the following sub-steps: S5-1, Establish the mapping relationship between product use and defect weight, and preset different defect category weights for automotive outer panels, automotive inner panels, home appliance panels and general industrial panels respectively; S5-2 divides the strip steel into edge region and middle region along the width direction, calculates the defect severity index and defect density in each region, and defines the edge defect weighting factor. When the defect is located in the edge region, the edge defect weighting factor is greater than 1, and when the defect is located in the middle region, the edge defect weighting factor is equal to 1. S5-3, Calculate the clustering effect factor of defects. When multiple defects are clustered in space, the clustering effect factor is greater than 1. When the defects are uniformly distributed, the clustering effect factor is equal to 1. S5-4 Calculate the overall quality score based on defect category weight, defect severity index, defect density, clustering effect factor, and edge defect weighting factor; S5-5 compares the overall quality score of the coil with the preset first and second scoring thresholds. When the overall quality score of the coil is less than or equal to the first scoring threshold, the coil of strip steel is determined to be Grade 1 and a qualified release instruction is output. When the overall quality score of the coil is greater than the first scoring threshold but less than the second scoring threshold, the coil of strip steel is determined to be Grade 2 and a downgrade usage instruction is output. When the overall quality score of the coil is greater than or equal to the second scoring threshold, the coil of strip steel is determined to be Grade 3 and a scrap or rework instruction is output, and an alarm is triggered.

[0162] Further, in sub-step S5-4, the specific formula for calculating the overall quality score is: Q=[Σ(q_i×S_i)+γ×(D_ensity / D_0)]×F_cluster×F_edge, where q_i is the defect category weight, S_i is the defect severity index, D_ensity is the defect density, D_0 is the reference density, F_cluster is the clustering effect factor, F_edge is the edge defect weighting factor, and γ is the density penalty coefficient.

[0163] It should be noted that the edge region refers to specific areas on both sides of the strip width direction. The impact of defects in this region on the subsequent processing and use of the strip differs from that in the middle region and requires separate weighted evaluation.

[0164] The middle area refers to the core area in the width direction of the strip after removing the edge areas. It is the main area for subsequent processing and use of the strip and is the core area of ​​focus for quality evaluation.

[0165] The edge defect weighting factor refers to the weighting coefficient set for defects in the edge area of ​​the strip, which is used to enhance the influence of edge area defects on the overall quality score of the strip.

[0166] The aggregation effect factor is a weighted coefficient that characterizes the spatial density of defects on the surface of strip steel, reflecting the additional impact of defect aggregation on strip steel quality.

[0167] The first scoring threshold refers to the critical value for judging strip steel as a first-grade product, and it is a quantitative standard for distinguishing between first-grade and second-grade products.

[0168] The second scoring threshold refers to the critical value for judging the strip steel as a grade three product, which is a quantitative standard for distinguishing between grade two and grade three products.

[0169] The density penalty coefficient is a coefficient used to adjust the weight of defect density in the overall coil quality score calculation, and is used to enhance the influence of defect distribution density on strip quality.

[0170] Reference density refers to the defect density reference value preset according to the quality standard of strip steel products, and is the benchmark value for defect density normalization calculation.

[0171] In one specific embodiment, the intelligent surface quality judgment method for cold-rolled strip steel provided by the present invention is applied to a 1450 mm cold-rolled continuous annealing production line of a steel enterprise. The production line rolls strip steel with a width range of 700 mm to 1450 mm and a rolling speed of 30 m / min to 120 m / min. It mainly produces four types of cold-rolled strip steel products: automotive outer panels, automotive inner panels, appliance panels, and ordinary industrial panels. Through previous steps, full data such as defect category, defect severity index, defect location, and defect density of the entire coil of strip steel have been obtained.

[0172] To address the production and quality assessment requirements of this production line, step S5-1 was first executed to establish a mapping relationship between product application and defect weights. Based on the quality standards for four product categories, differentiated weights were assigned to five common defects: scratches, pitting, roller marks, scars, and inclusions. For automotive outer panels, the weights for scars and inclusions were set to 0.9, scratches and roller marks to 0.7, and pitting to 0.5; for automotive inner panels, the weights for scars and inclusions were set to 0.7, scratches and roller marks to 0.6, and pitting to 0.4; for appliance panels, the weights for scars and inclusions were set to 0.6, scratches and roller marks to 0.5, and pitting to 0.3; and for general industrial panels, all defects were assigned a weight of 0.3, thus achieving personalized weight settings based on product application.

[0173] Next, step S5-2 is executed, defining 15% on each side of the strip width as the edge region and the remaining 70% as the middle region. For the 1200 mm wide strip currently being rolled on this production line, 180 mm on each side is the edge region and 840 mm in the middle is the middle region. The defect severity index and defect density of the edge and middle regions are calculated separately. A defect weighting factor F_edge is set for the edge region, F_edge for the middle region defects is set to 1.0, and F_edge for the edge region defects is set to 1.5, thus strengthening the quality impact weight of edge defects.

[0174] Then, perform step S5-3 to calculate the clustering effect factor F_cluster of defects. Using 100 mm × 100 mm as a statistical unit, when the number of defects in the unit is less than 3, it is judged as uniform distribution, and F_cluster is set to 1.0; when the number of defects in the unit is 3 to 5, it is judged as slight clustering, and F_cluster is set to 1.3; when the number of defects in the unit is greater than 5, it is judged as severe clustering, and F_cluster is set to 1.8, thus quantifying the quality impact of defect clustering distribution.

[0175] Then, step S5-4 is executed, using the formula Q=[Σ(q_i×S_i)+γ×(D_ensity / D_0)]×F_cluster×F_edge to calculate the overall roll quality score. According to the production line quality standards, the preset reference density D_0 is 5 per square meter, and the density penalty coefficient γ is 2.0. First, the product of the category weight and severity index of each type of defect is calculated, Σ(q_i×S_i). Then, the ratio of defect density to reference density is calculated and multiplied by the density penalty coefficient. The two results are added together and then multiplied by the clustering effect factor and the edge defect weighting factor to obtain the final overall roll quality score.

[0176] Finally, step S5-5 is executed. Based on the quality requirements of the four product categories, preset scoring thresholds are set: For automotive outer panels, the first scoring threshold is 3.0, and the second is 6.0; for automotive inner panels, the first is 4.0, and the second is 7.0; for appliance panels, the first is 5.0, and the second is 8.0; and for general industrial panels, the first is 6.0, and the second is 9.0. The calculated quality score of the entire roll is compared with the corresponding product thresholds. If the score is less than or equal to the first scoring threshold, it is classified as a Grade 1 product and a qualified release instruction is output. If the score is greater than the first scoring threshold but less than the second scoring threshold, it is classified as a Grade 2 product and a downgrade instruction is output. If the score is greater than or equal to the second scoring threshold, it is classified as a Grade 3 product and a scrap or rework instruction is output. Simultaneously, an audible and visual alarm is triggered on the production line to remind staff to handle the situation promptly. In its practical application on the 1450 mm cold rolling continuous annealing production line, this quality assessment method has enabled refined quality grading of strip steel for different purposes. The matching degree between quality assessment and actual product usage requirements has been improved to over 99%, effectively reducing the downgrading of high-quality strip steel and the release of inferior strip steel, thereby improving the product quality control level and production efficiency of the production line.

[0177] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for intelligently determining the surface quality of cold-rolled strip steel based on image processing, characterized in that, Includes the following steps: Step S1: A line scan camera is used in conjunction with a composite light source that alternates between bright and dark fields to alternately acquire bright field images and dark field images of the steel strip surface. The two images are then fused to eliminate the local information loss caused by the reflection of the metal mirror and obtain a preprocessed image. Step S2: For the background grayscale fluctuation caused by the rolling texture on the surface of cold-rolled strip steel, a local dynamic threshold with the window size dynamically changing with the background texture period is used to segment the preprocessed image, and the real defect area is obtained through morphological screening. Step S3: Extract the geometric features of the real defect area, the texture features based on the gray-level co-occurrence matrix, and the frequency domain features reflecting the repetition period of the defect to construct a multi-dimensional feature vector. Step S4: Identify the defect category based on multidimensional feature vectors, reconstruct the three-dimensional morphology of the defect based on the photometric difference between the bright field image and the dark field image, and fuse the two-dimensional geometric parameters and three-dimensional depth parameters of the defect to calculate the defect severity index. Step S5: Establish a configurable defect weighting system that is linked to the product's intended use. Calculate the overall roll quality score based on defect category weights, defect severity index, and defect density, and automatically output the quality level.

2. The intelligent surface quality determination method for cold-rolled strip steel based on image processing according to claim 1, characterized in that: Step S1 further includes the following sub-steps: S1-1, using a preset isochronous alternation method, controls the line scan camera to acquire the first set of images in conjunction with bright field light source and dark field light source, forming the first bright field image and the first dark field image respectively; S1-2, Calculate the gradient direction histogram of the first bright field image and the first dark field image, and extract the main peak direction as the rolling texture direction; S1-3, Determine the ratio of bright field acquisition time to dark field acquisition time based on the angle between the rolling texture direction and the light source illumination direction; S1-4, according to a determined time ratio, control the linear array camera to collect images of the strip surface in conjunction with bright field light source and dark field light source. In each scanning cycle, first collect the original bright field image according to the bright field acquisition time, and then collect the original dark field image according to the dark field acquisition time. The process is repeated alternately to form the original bright field image and the original dark field image respectively. S1-5, calculate the gray-level variance of each pixel in the preset neighborhood window of the whole bright field original image and the whole dark field original image respectively, and obtain the bright field variance map and the dark field variance map. S1-6, Determine the relative magnitudes of the bright-field blending coefficient and the dark-field blending coefficient based on the angle between the rolling texture direction and the light source illumination direction; S1-7. According to the determined fusion coefficient, the original bright field image and the original dark field image are fused at the pixel level to generate a preprocessed image. The fusion formula is: I_fusion=w×I_bright+(1-w)×I_dark, where I_fusion is the preprocessed image, I_bright is the original bright field image, I_dark is the original dark field image, and w is the fusion coefficient of the bright field image.

3. The intelligent surface quality determination method for cold-rolled strip steel based on image processing according to claim 2, characterized in that, In sub-steps S1-3, the ratio of the bright-field acquisition time to the dark-field acquisition time is determined according to the following formula: When the angle between the rolling texture direction and the light source illumination direction is less than the preset angle threshold, the bright field acquisition time: dark field acquisition time = tan(angle): 1; When the included angle is equal to the preset included angle threshold, the ratio of bright field acquisition time to dark field acquisition time is 1:

1. When the included angle is greater than the preset included angle threshold, the bright field acquisition time : dark field acquisition time = 1 : tan(90° - included angle).

4. The intelligent surface quality determination method for cold-rolled strip steel based on image processing according to claim 2, characterized in that, In sub-steps S1-6, the relative magnitudes of the bright-field fusion coefficient and the dark-field fusion coefficient are determined according to the following formula: When the angle between the rolling texture direction and the light source illumination direction is less than the preset angle threshold, the bright field fusion coefficient = 1 / (1+tan(angle)). When the included angle is equal to the preset included angle threshold, the bright field fusion coefficient is 0.5; When the included angle is greater than the preset included angle threshold, the bright field fusion coefficient = tan(included angle) / (1 + tan(included angle)).

5. The intelligent surface quality determination method for cold-rolled strip steel based on image processing according to claim 1, characterized in that: Step S2 further includes the following sub-steps: S2-1, Based on the determined rolling texture direction, the window shape of the local dynamic threshold is set to a rectangular window that is stretched along this direction, with the window length direction parallel to the texture direction and the window width direction perpendicular to the texture direction; S2-2, Based on the period length P of the rolling texture, set the length L of the rectangular window to m×P and the width W to n×P, where m≥2 and n≤1; S2-3, within the defined rectangular window, calculate the dynamic segmentation threshold, specifically using the formula T(x,y) = μ(x,y). (x, y), where μ(x, y) is the mean gray value within the window, σ(x, y) is the standard deviation of gray values ​​within the window, and k is an adaptive coefficient that dynamically adjusts with the variance of gray values ​​within the local window; S2-4: Mark pixels with gray values ​​lower than the dynamic segmentation threshold T(x,y) as candidate defect points to obtain the binarized defect region; S2-5, Morphological processing is performed on the binarized defect area. First, a structuring element that is elongated along the texture direction is used to perform a closing operation, and then a circular structuring element is used to perform an opening operation. S2-6. Perform connected component analysis on the processed binarized image, and set an adaptive area filtering threshold based on the aspect ratio of the connected components to obtain the real defect area.

6. The intelligent surface quality determination method for cold-rolled strip steel based on image processing according to claim 1, characterized in that: Step S3 further includes the following sub-steps: S3-1, calculate the area, perimeter, aspect ratio of the circumscribed rectangle, and roundness of the actual defect region, and calculate the relative direction angle with the rolling texture direction as a reference, as well as the ratio of the length of the defect along the rolling direction to the length perpendicular to the rolling direction, to form a geometric feature vector. The relative direction angle is the difference between the defect direction angle and the rolling texture direction. S3-2, construct gray-level co-occurrence matrices along the rolling texture direction, perpendicular to the rolling texture direction, and diagonal direction respectively, calculate the energy, entropy, moment of inertia, and correlation of each direction, and weight and fuse the feature parameters of the three directions according to preset weights to form a texture feature vector; S3-3, Perform a two-dimensional Fourier transform on the real defect area, expand the spectrum in polar coordinates, extract the radial energy distribution and angular energy distribution, and form a frequency domain feature vector; S3-4 concatenates the geometric feature vector, texture feature vector, and frequency domain feature vector to construct a multidimensional feature vector.

7. The intelligent surface quality determination method for cold-rolled strip steel based on image processing according to claim 1, characterized in that: Step S4 further includes the following sub-steps: S4-1, input the multidimensional feature vector into the lightweight convolutional neural network and output the defect category probability distribution. The lightweight convolutional neural network adopts depthwise separable convolution and channel attention mechanism. S4-2, using the entire bright field original image and the entire dark field original image, extract the grayscale value of the same point in the defect area under bright field illumination and dark field illumination; S4-3 uses the photometric stereo method to solve for the normal vector, and introduces the specular reflection component model in the solution. The bright field image is used as the diffuse reflection component and the dark field image is used as the specular reflection component. The normal vector is solved after separating the diffuse reflection component and the specular reflection component. S4-4, integrate and reconstruct the normal vector field obtained by the solution to obtain a three-dimensional depth map of the defect area, and extract the average depth H_est and depth standard deviation H_std of the defect. S4-5, Measure the projected length of the defect along the rolling direction and the maximum width perpendicular to the rolling direction, and count the number of defects per unit area. S4-6, Calculate the defect severity index using the following formula: S = α × (R / R_0) × (K / K_0) × (H_est / H_0) × (E / E_0) × (1 + β × H_std / H_est), where R is the projected length of the defect along the rolling direction, R_0 is the reference value for the projected length, K is the maximum width of the defect perpendicular to the rolling direction, K_0 is the reference value for the maximum width, H_est is the average depth of the defect, H_0 is the reference value for the depth, E is the number of defects per unit area, E_0 is the reference value for the number, H_std is the standard deviation of the defect depth, α is the normalization coefficient, and β is the depth uniformity adjustment coefficient.

8. The intelligent surface quality determination method for cold-rolled strip steel based on image processing according to claim 7, characterized in that, In sub-step S4-3, the step of solving the normal vector after separating the diffuse reflection component and the specular reflection component includes: The grayscale values ​​of the dark field image are used as the specular reflection component, and the specular reflection normal vector is solved by the photometric stereo method. Subtracting the gray values ​​of the dark field image from the gray values ​​of the bright field image yields the diffuse reflection component. Based on the diffuse reflection component, the diffuse reflection normal vector is solved using the photometric stereo method. The specular reflection normal vector and the diffuse reflection normal vector are weighted and fused to obtain the final normal vector of the defective surface.

9. The intelligent surface quality determination method for cold-rolled strip steel based on image processing according to claim 1, characterized in that: Step S5 further includes the following sub-steps: S5-1, Establish the mapping relationship between product use and defect weight, and preset different defect category weights for automotive outer panels, automotive inner panels, home appliance panels and general industrial panels respectively; S5-2 divides the strip steel into edge region and middle region along the width direction, calculates the defect severity index and defect density in each region, and defines the edge defect weighting factor. When the defect is located in the edge region, the edge defect weighting factor is greater than 1, and when the defect is located in the middle region, the edge defect weighting factor is equal to 1. S5-3, Calculate the clustering effect factor of defects. When multiple defects are clustered in space, the clustering effect factor is greater than 1. When the defects are uniformly distributed, the clustering effect factor is equal to 1. S5-4 Calculate the overall quality score based on defect category weight, defect severity index, defect density, clustering effect factor, and edge defect weighting factor; S5-5 compares the overall quality score of the coil with the preset first and second scoring thresholds. When the overall quality score of the coil is less than or equal to the first scoring threshold, the coil of strip steel is determined to be Grade 1 and a qualified release instruction is output. When the overall quality score of the coil is greater than the first scoring threshold but less than the second scoring threshold, the coil of strip steel is determined to be Grade 2 and a downgrade usage instruction is output. When the overall quality score of the coil is greater than or equal to the second scoring threshold, the coil of strip steel is determined to be Grade 3 and a scrap or rework instruction is output, and an alarm is triggered.

10. The intelligent method for determining the surface quality of cold-rolled strip steel based on image processing according to claim 9, characterized in that, In sub-step S5-4, the specific formula for calculating the overall quality score is: Q=[Σ(q_i×S_i)+γ×(D_ensity / D_0)]×F_cluster×F_edge, where q_i is the defect category weight, S_i is the defect severity index, D_ensity is the defect density, D_0 is the reference density, F_cluster is the clustering effect factor, F_edge is the edge defect weighting factor, and γ is the density penalty coefficient.

Images