Deep learning-based blank surface oxide skin proportion identification method and system

Abstract

The invention relates to the technical field of image recognition, in particular to a blank surface oxide skin proportion recognition method and system based on deep learning, and the method comprises the following steps: obtaining effective data through a plurality of preprocessing algorithms such as bilateral filtering, and recognizing the boundary of a product through a deep learning fine tuning model dynamic boundary detection method; a complete side surface of the blank is obtained by using a matching splicing algorithm and cutting splicing, finally, a picture is converted into a grey-scale map according to the characteristics of the scale scales, and the proportion of the scale scales with different thicknesses on the side surface of the blank is identified by adopting a brightness weighting mode. According to the method, the product boundary is obtained through dynamic boundary detection of the deep learning fine tuning model, and the multiple frames of local images are spliced and cut into the complete blank side expanded view based on the matching splicing algorithm, so that reconstruction of complete surface information is realized, and the defect that the whole surface of a large-size workpiece cannot be covered by a single view angle is overcome; and meanwhile, accurate data support is provided for subsequent oxide skin identification.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a method and system for identifying the percentage of oxide scale on the surface of billets based on deep learning. Background Technology

[0002] Extrusion, as one of the core technologies in the field of metal plastic forming, has become a core forming method for metal components in aerospace, petrochemical, and other fields due to its "near-net-shape" advantage. Quality control in extrusion not only affects product qualification rates but also relates to the operational safety of end-use equipment. The "oxide scale ratio," as a core indicator of billet surface quality, is a crucial link in the quality control of the extrusion process, and its accurate detection is essential. If the oxide scale (mainly composed of metal oxides) on the billet surface is not effectively controlled during extrusion, it will impair the mechanical properties of the product, directly affecting the forming effect and quality, reducing forming accuracy, and also exacerbating die wear and increasing production costs.

[0003] Currently, oxide scale detection relies heavily on manual methods, which suffers from high subjectivity, low efficiency, and susceptibility to environmental interference. Existing technologies, such as CN118038178A (Classification of Defects in Hot-Rolled Strip Steel), involve image recognition and utilize a fixed-step window method to perform frame-by-frame detection on newly acquired hot-rolled strip steel images to efficiently and accurately identify surface defects in newly acquired images. However, they cannot specifically stitch together and unfold video frames around the perimeter of the metal column, nor can they solve problems such as data cleaning, boundary detection, and image stitching under interference from flames, glass powder, etc., nor can they achieve quantitative analysis of oxide scale thickness ratio.

[0004] Therefore, existing technologies cannot specifically handle the scenario of video frame splicing and unfolding around the metal column, nor can they solve the core defects such as preprocessing, boundary detection, and overlap fusion in this scenario, resulting in poor accuracy in recognizing the proportion of oxide scale on the billet surface. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for identifying the proportion of oxide scale on the surface of billets based on deep learning, so as to solve the technical problems of existing technologies that specifically process video frame splicing and unfolding scenarios around the metal column, and cannot solve the core defects such as preprocessing, boundary line detection, and overlap fusion in such scenarios.

[0006] To solve the above-mentioned technical problems, the present invention specifically provides the following technical solution: A deep learning-based method for identifying the oxide scale ratio on the surface of billets includes the following steps: S1: Based on the specifications of the billet product, control the camera to adaptively adjust the shooting position and collect video of the outer circumference surface of the billet around the entire circumference, adapting to the billet rotation speed. S2: Extract multiple valid video frames from the video of the outer peripheral surface, and perform preprocessing such as bilateral filtering for noise reduction, CLAHE contrast enhancement and Laplacian sharpening in sequence to obtain preprocessed video frames; S3: Based on a deep learning model, identify the blank boundary line in the preprocessed video frame, and perform anomaly detection and interpolation correction on the boundary line identification results; S4: Based on the feature matching algorithm, the preprocessed video frames are stitched together into a billet outer peripheral surface unfolding diagram using boundary lines. Based on NCC verification, the billet outer peripheral surface unfolding diagram is stitched together again to make up for the lack of viewpoints in unidirectional unfolding, so as to obtain a complete billet outer peripheral surface unfolding diagram. S5: Identify the pixel ratio of oxide scale flakes of different thicknesses on the surface of the billet in the unfolded image of the complete outer peripheral surface of the billet based on the brightness weighting method.

[0007] As a preferred embodiment of the present invention, in step S2, the method for extracting multiple valid video frames from the video of the outer peripheral surface includes: Select the start extraction time Frame interval k, number of target extraction frames N, rotation angle The starting frame index is calculated by combining the video frame rate (fps). With the set of frame indices to be extracted ,in For floor operations, if If so, it will be automatically truncated to the total number of frames; In different shooting scenarios, through rotation matrix Translation vector Rotate all frames to be extracted to a uniform orientation, and then determine the pixel coordinates of the video frames. The coordinate transformation relationship with the original coordinates (x, y) is as follows: in It consists of the original frame width W and the original frame height H of the video frame; In the set of frame indexes to be extracted Extract M video frames to form a list of valid video frames .

[0008] As a preferred embodiment of the present invention, the preprocessing method in S2, which includes bilateral filtering denoising, CLAHE contrast enhancement, and Laplacian sharpening, comprises: Set the filter diameter d and the color standard deviation Spatial standard deviation Set to noise reduction intensity Linear correlation involves performing bilateral filtering on valid video frames, resulting in the effective video frame pixels after bilateral filtering. in For the neighborhood, These are Gaussian functions for the color domain and the spatial domain, respectively. The normalization coefficient is... Center pixel pixel intensity, For the neighborhood medium pixel Pixel intensity; The effective video frames after bilateral filtering are converted to the YCrCb color space, the luminance channel Y is extracted, and the contrast is enhanced using the CLAHE algorithm to obtain the enhanced luminance channel. Then, the Cr and Cb channels are merged and converted back to the BGR color space to obtain the effective video frames after CLAHE contrast enhancement. Due to cropping limitations, Determine the number of grid divisions; By comparing with sharpening intensity Associated convolutional kernels Effective video frames after CLAHE contrast enhancement Perform convolution enhancement to obtain sharpened pixel values. And sharpened pixel values After truncation and limiting the value to the range of 0-255, the effective video frames after Laplacian sharpening are obtained. As preprocessed video frames, For convolution operations, This is a truncation function.

[0009] As a preferred embodiment of the present invention, in step S3, the method for identifying the blank boundary line in the preprocessed video frame based on a deep learning model and performing anomaly detection and interpolation correction on the boundary line identification result includes: The preprocessed video frames are normalized, channel-converted, and pixel-value-standardized according to a preset size to obtain a tensor of the image to be detected that is adapted to the input of the deep learning model. This tensor is then input into a billet boundary line detection model pre-trained based on the YOLOv12 single-stage object detection algorithm. The training process of this boundary line detection model is as follows: using video frame images with labeled billet boundary line regions as training samples, a billet boundary line detection dataset containing training, validation, and test sets is constructed. Feature extraction is performed based on the backbone network of the YOLOv12 algorithm. Its neck network (SPPF spatial pyramid pooling structure) fuses multi-scale feature maps, and then the detection head network performs bounding box regression and class confidence prediction on the fused feature maps. The CIoU loss function is used to optimize the bounding box regression accuracy, and the cross-entropy loss function is used to optimize the class classification accuracy. After multiple rounds of iterative training and validation set tuning, a convergent boundary detection model with detection accuracy meeting the preset threshold is obtained. In the inference stage, after feature extraction and target detection on the input image tensor, the model outputs the predicted bounding box parameters representing the position of the billet boundary line, denoted as the bounding box. ,in The coordinates of the top left corner of the bounding box. This represents the coordinates of the lower right corner of the bounding box; the model will also output the detection confidence of the corresponding blank boundary line for subsequent selection of valid bounding boxes.

[0010] For the detection results of a single frame of a valid video frame, select the bounding box with the highest confidence. Take the bounding box The average of the upper and lower y-coordinates is used as the upper boundary line. ,in , Boundary frames Upper and lower y-coordinates; Summarize the boundary lines of all valid video frames, and fill the boundary lines of valid video frames that failed bounding box detection with 0s to obtain a boundary line list. ,in , These are the boundary lines of the 1st and Mth valid video frames, respectively; Calculate the median separately absolute deviation MAD value ,in Let i be the boundary line of the i-th valid video frame. , and represent the absolute deviations of the i-th and M-th valid video frames, respectively. For the median operator, This is the absolute value operator; Will corresponding Mark outliers as and the remaining corresponding Mark non-outlier values ​​as , Collect the indices and values ​​of non-outliers And for each outlier Find the nearest non-outlier values ​​on the left and right, and then check the outlier values. Based on distance-weighted interpolation processing ,in , These are outliers The nearest non-outlier values ​​on the left and right sides , They are respectively , The index is filled with the median of non-outliers when no interpolation is available; List of dividing lines After interpolation, it is converted into a new list of boundary lines. .

[0011] As a preferred embodiment of the present invention, in step S4, the method of stitching preprocessed video frames into a unfolded diagram of the outer periphery of the billet using boundary lines based on a feature matching algorithm includes: Bottom cropping is performed sequentially on the preprocessed video frames. Horizontal cropping To obtain the effective width Where H is the frame height. This refers to the bottom cutting ratio. ; The SIFT detector is used to extract the previous frame of the preprocessed video frame. With the current frame Feature points , descriptor Using a brute-force matcher (BFMatcher) Perform K-nearest neighbor (KNN) matching to select a set of high-quality matching feature points. Where m is the nearest neighbor match and n is the second nearest neighbor match. For the closest distance, The second closest distance, ; If matching point 10. Calculate the homography matrix H using RANSAC robust estimation, and then use the previous frame... Projecting the four corners onto the current frame, calculating the maximum projected x-coordinate. The previous frame before projection The overlap width at the four corners , if matching point Using a preset overlap width And by setting a preset overlap width limit, the overlap width after the limit is obtained by avoiding extreme values. ; Initialize a white background (255, 255, 255), and then use Gaussian weighted blending to obtain the stitched unfolded image. The average overlap width is obtained as Unfolded diagram with white background fill As the unfolded diagram of the outer perimeter of the billet, where I The corresponding image dimensions [rows, columns, channels], Gaussian weights , This is a weighted fusion function, where c is the column index within the overlapping region. This represents the starting x-coordinate of the current frame in the unfolded graph. The overlap width after limitation in the i-th video frame .

[0012] As a preferred embodiment of the present invention, the method for secondary splicing of the unfolded view of the outer periphery of the billet based on NCC verification to compensate for the lack of perspective in unidirectional unfolding in S4 includes: Take the minimum height of the left and right portions of the unfolded outer circumference of the blank, and cut it to a uniform height. ,in These represent the heights of the left and right sections, respectively. To perform the minimum operation; After unifying the height, the left and right portions of the unfolded outer perimeter of the billet are subjected to grayscale conversion and histogram equalization to obtain... , ,in , These are the left and right portions of the unfolded view of the outer perimeter of the billet after it has been standardized to a uniform height. This is a grayscale identifier. This is a color conversion function. For histogram equalization function, , These are grayscale images of the left and right portions of the image after conversion and equalization, respectively. Using SIFT estimates Calculate the candidate overlap width range ; NCC verification calculates the normalized cross-correlation coefficient. And select the one with the largest NCC value. As the optimal overlap width ,in For covariance, Standard deviation; Calculate the total width of the splicing Gaussian weighted fusion is applied to the overlapping areas, with the Gaussian weight of column c being... The corresponding columns of the left and right images are merged according to weights (1-α) and α to obtain the complete stitched image. As a complete unfolded view of the outer perimeter of the billet, in which , These represent the widths of the left and right portions of the unfolded diagram of the outer perimeter of the billet.

[0013] As a preferred embodiment of the present invention, in step S5, the method for identifying the pixel ratio of oxide scale flakes of different thicknesses on the surface of the billet in the unfolded image of the complete outer peripheral surface of the billet based on the brightness weighting method includes: Traverse the complete outer perimeter development of the billet column by column, exclude the continuous pure white (255,255,255) background, record the effective pixel range of each column, and count the total number of effective pixels; Based on the radiation characteristics of high-temperature metals, the brightness value of each effective pixel is calculated. ,in , , R, G, and B are the channel values, respectively. The pixels are divided into multiple intervals based on the brightness value. Each interval corresponds to a different oxide thickness level. The number of pixels in each interval is counted and the proportion of the total effective pixels is calculated to obtain the thickness distribution ratio of the oxide scales.

[0014] As a preferred embodiment of the present invention, the present invention provides a billet surface oxide scale ratio recognition system based on deep learning, applied to a billet surface oxide scale ratio recognition method based on deep learning, the system comprising: The data acquisition unit is used to control the camera to adaptively adjust the shooting position according to the specifications of the billet product, and to acquire video of the outer circumference surface of the billet by adapting to the billet rotation speed. The preprocessing unit is used to extract multiple valid video frames from the video of the outer peripheral surface, and sequentially perform bilateral filtering noise reduction, CLAHE contrast enhancement and Laplacian sharpening preprocessing to obtain preprocessed video frames; The boundary recognition unit is used to identify the blank boundary line in the preprocessed video frame based on a deep learning model, and to perform anomaly detection and interpolation correction on the boundary line recognition results. The image stitching unit is used to stitch preprocessed video frames into a billet outer peripheral surface unfolded image based on the feature matching algorithm using boundary lines, and to perform secondary stitching on the billet outer peripheral surface unfolded image based on NCC verification to make up for the lack of viewpoints in unidirectional unfolding, so as to obtain a complete billet outer peripheral surface unfolded image. The proportion analysis unit identifies the pixel proportion of oxide scale flakes of different thicknesses on the surface of the billet in the unfolded image of the complete outer peripheral surface of the billet based on the brightness weighting method.

[0015] As a preferred embodiment of the present invention, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement a method for identifying the proportion of oxide scale on the surface of a billet based on deep learning.

[0016] Compared with the prior art, the present invention has the following advantages: This invention uses camera position control and rotation speed adaptation acquisition logic to cover cylindrical blanks of different diameters and lengths, eliminating the need for manual adjustment of shooting parameters, adapting to the dynamic production rhythm of the workshop, and reducing operating costs.

[0017] This invention addresses interference from workshop flames, glass dust, and uneven brightness by employing a combination of bilateral filtering, CLAHE contrast enhancement, and Laplacian sharpening preprocessing to effectively remove noise and enhance oxide scale texture details, thereby improving the signal-to-noise ratio of the preprocessed image.

[0018] This invention uses a deep learning fine-tuning model to dynamically detect product boundaries and uses a matching stitching algorithm to stitch and crop multiple frames of local images into a complete unfolded view of the blank side, thus reconstructing complete surface information. This solves the problem that a single view cannot cover the entire surface of a large workpiece and provides accurate data support for subsequent oxide scale identification. Attached Figure Description

[0019] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0020] Figure 1 A flowchart of a method for identifying the percentage of oxide scale on the surface of a billet provided in an embodiment of the present invention; Figure 2 This is a boundary recognition result image of a single preprocessed image provided in an embodiment of the present invention; Figure 3 This is a flowchart combining the left and right portions of the unfolded view of the outer periphery of the billet provided in an embodiment of the present invention; Figure 4 This is a fully unfolded view of the outer periphery of the billet provided in an embodiment of the present invention; Figure 5 This is a diagram showing the percentage of oxide scales of different thicknesses provided in an embodiment of the present invention; Figure 6 A block diagram of a billet surface oxide scale ratio identification system provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] like Figure 1 As shown, this invention provides a method for identifying the percentage of oxide scale on the surface of billets based on deep learning, comprising the following steps: S1: Based on the specifications of the billet product, control the camera to adaptively adjust the shooting position and collect video of the outer circumference surface of the billet around the entire circumference, adapting to the billet rotation speed. S2: Extract multiple valid video frames from the video of the outer surface, and perform preprocessing such as bilateral filtering for noise reduction, CLAHE contrast enhancement and Laplacian sharpening in sequence to obtain preprocessed video frames; S3: Based on a deep learning model, identify the blank boundary line in the preprocessed video frame, and perform anomaly detection and interpolation correction on the boundary line identification results; S4: Based on the feature matching algorithm, the preprocessed video frames are stitched together to form the unfolded image of the outer periphery of the billet. The unfolded image is then cropped using the boundary lines to eliminate irrelevant backgrounds. Based on NCC verification, the unfolded image of the outer periphery of the billet is stitched together again to compensate for the lack of viewpoints in unidirectional unfolding, thus obtaining the complete unfolded image of the outer periphery of the billet. S5: Identify the pixel ratio of oxide scale flakes of different thicknesses on the surface of the billet in the unfolded image of the complete outer peripheral surface of the billet based on the brightness weighting method.

[0023] This invention employs a processing flow of "video acquisition - interference removal - boundary recognition - image stitching - proportion recognition," utilizing various preprocessing algorithms such as bilateral filtering to obtain effective data. It identifies product boundaries through a deep learning-based fine-tuned model dynamic boundary detection method, and uses a matching stitching algorithm to crop and stitch together the complete side surface of the billet. Finally, based on the characteristics of oxide scale, the image is converted into a grayscale image, and a brightness-weighted approach is used to identify the proportion of oxide scale of different thicknesses on the side surface of the billet.

[0024] To acquire a complete image of the outer peripheral surface of a blank product, this invention first uses a control system in the data acquisition stage to automatically move the camera to a predetermined position based on the product's specifications. This ensures that the distance between the object and the camera is within the range of the depth of field and near-depth of field, thereby acquiring effective and clear product surface data. The acquisition duration is automatically determined based on the product's rotation speed, ensuring that the acquisition time covers the time it takes for the product to complete one rotation. The acquired video data provides the data foundation and serves as a basis for subsequent stitching and cropping of complete product surface images. From the video of the product's peripheral surface, a specified number of valid frames are extracted at set intervals and corrected according to the shooting angle rotation, as detailed below: In S2, the method for extracting multiple valid video frames from the video of the outer peripheral surface includes: Select the start extraction time Units: s, frame interval k, frames, number of target extraction frames N, rotation angle Unit: °, Calculate the starting frame index based on the video frame rate (fps). With the set of frame indices to be extracted ,in For floor operations, if If so, it will be automatically truncated to the total number of frames; In different shooting scenarios, through rotation matrix Translation vector Rotate all frames to be extracted to a uniform orientation, and then determine the pixel coordinates of the video frames. The coordinate transformation relationship with the original coordinates (x, y) is as follows: in It consists of the original frame width W and the original frame height H of the video frame; In the set of frame indexes to be extracted Extract M video frames to form a list of valid video frames .

[0025] Specifically, for horizontal shooting scenarios... , , Through rotation matrix Implement video frame rotation, and determine the pixel coordinates of the rotated frame. The coordinate transformation relationship with the original coordinates (x, y) is as follows: .

[0026] For example, , , .

[0027] Collecting data during the product manufacturing process allows for more efficient and timely data acquisition without disrupting the normal workflow, but it faces interference from flames and glass dust. To address this, this invention employs bilateral filtering to remove interference, CLAHE to balance brightness unevenness caused by flames and enhance details in dust-covered areas, Laplacian operator to restore blurred edges from dust, and SIFT+RANSAC feature matching to filter out abnormal matches, ensuring the stability of subsequent image stitching and other processing. To avoid interference from complex workshop lighting environments and temperature differences between different products, the data is preprocessed and key features such as contrast are enhanced to improve the accuracy of subsequent feature matching, as detailed below: In S2, the preprocessing methods include bilateral filtering for noise reduction (which can be replaced with Gaussian filtering, resulting in improved overall speed but reduced texture preservation), CLAHE contrast enhancement, and Laplacian sharpening: The first step is bilateral filtering for noise reduction: removing high-frequency noise while preserving the texture features of the metal surface.

[0028] Set the filter diameter d=9 and the color standard deviation Spatial standard deviation Set to noise reduction intensity Linear correlation involves performing bilateral filtering on valid video frames, resulting in the effective video frame pixels after bilateral filtering. in For the neighborhood, These are Gaussian functions for the color domain and the spatial domain, respectively. The normalization coefficient is... Center pixel pixel intensity, For the neighborhood medium pixel Pixel intensity; The second step involves CLAHE to balance the uneven brightness caused by the flame and enhance details in areas covered by dust.

[0029] The effective video frames after bilateral filtering are converted to the YCrCb color space, the luminance channel Y is extracted, and the contrast is enhanced using the CLAHE algorithm to obtain the enhanced luminance channel. Then, the Cr and Cb channels are merged and converted back to the BGR color space to obtain the effective video frames after CLAHE contrast enhancement. Due to cropping limitations, Determine the number of grid divisions; When the effective video frame is a grayscale frame, the CLAHE algorithm is applied directly to the grayscale image. The third step is to use the kernel. Enhance the texture details of metal surfaces and output Add restrictions to ensure pixel values ​​are valid.

[0030] By comparing with sharpening intensity Associated convolutional kernels Effective video frames after CLAHE contrast enhancement Perform convolution enhancement to obtain sharpened pixel values. And sharpened pixel values After truncation and limiting the value to the range of 0-255, the effective video frames after Laplacian sharpening are obtained. As preprocessed video frames, For convolution operations, This is a truncation function.

[0031] In the high-quality video frames obtained after preprocessing, this invention uses a deep learning fine-tuning model to dynamically detect product boundaries. This involves detecting and correcting failed or abnormal boundary lines (such as those caused by motion-blurred frames) to ensure the continuity of the subsequent unfolded region. Specifically: In S3, the method for identifying billet boundary lines in preprocessed video frames based on a deep learning model, and performing anomaly detection and interpolation correction on the boundary line identification results, includes: The preprocessed video frames are normalized, channel-converted, and pixel-value-standardized according to a preset size to obtain a tensor of the image to be detected that is adapted to the input of the deep learning model. This tensor is then input into a billet boundary line detection model pre-trained based on the YOLOv12 single-stage object detection algorithm. The training process of this boundary line detection model is as follows: using video frame images with labeled billet boundary line regions as training samples, a billet boundary line detection dataset containing training, validation, and test sets is constructed. Feature extraction is performed based on the backbone network of the YOLOv12 algorithm. Its neck network (SPPF spatial pyramid pooling structure) fuses multi-scale feature maps, and then the detection head network performs bounding box regression and class confidence prediction on the fused feature maps. The CIoU loss function is used to optimize the bounding box regression accuracy, and the cross-entropy loss function is used to optimize the class classification accuracy. After multiple rounds of iterative training and validation set tuning, a convergent boundary detection model with detection accuracy meeting the preset threshold is obtained. In the inference stage, after feature extraction and target detection on the input image tensor, the model outputs the predicted bounding box parameters representing the position of the billet boundary line, denoted as the bounding box. ,in The coordinates of the top left corner of the bounding box. The coordinates of the bottom right corner of the bounding box; and also as Figure 2 As shown, the model will also output the detection confidence of the blank boundary line corresponding to the bounding box, which will be used for subsequent screening of valid bounding boxes.

[0032] The YOLO algorithm for boundary line recognition can be replaced with the Canny detection algorithm and Hough transform. After the replacement, manual parameter tuning is required.

[0033] For the detection results of a single frame of a valid video frame, select the bounding box with the highest confidence. Take the bounding box The average of the upper and lower y-coordinates is used as the upper boundary line. ,in , Boundary frames Upper and lower y-coordinates; Summarize the boundary lines of all valid video frames, and fill the boundary lines of valid video frames that failed bounding box detection with 0s to obtain a boundary line list. ,in , These are the boundary lines of the 1st and Mth valid video frames, respectively; Calculate the median separately absolute deviation MAD value ,in Let i be the boundary line of the i-th valid video frame. , and represent the absolute deviations of the i-th and M-th valid video frames, respectively. For the median operator, This is the absolute value operator; Will corresponding Mark outliers as and the remaining corresponding Mark non-outlier values ​​as , Collect the indices and values ​​of non-outliers And for each outlier Find the nearest non-outlier values ​​on the left and right, and then check the outlier values. Based on distance-weighted interpolation processing ,in , These are outliers The nearest non-outlier values ​​on the left and right sides , They are respectively , The index is filled with the median of non-outliers when no interpolation is available; List of dividing lines After interpolation, it is converted into a new list of boundary lines. .

[0034] This invention uses the deep learning YOLO algorithm to detect the upper boundary line of a product: YOLO Model Loading: Load the pre-trained boundary detection model; the model outputs cylindrical bounding boxes. (Where x and y correspond to: top left x, top left y, bottom right x, bottom right y respectively); Boundary line calculation: For single-frame detection results, select the bounding box with the highest confidence. The average of the upper and lower y-coordinates of the bounding box is taken as the upper boundary line. , (y-coordinates) In other words, the input is a YOLO model and the frame to be detected; the output is the upper boundary of a single frame (integer pixel coordinates), and returns None if the detection fails.

[0035] This invention uses the median absolute deviation (MAD) method for outlier detection, and the specific steps are as follows: Let the original list of M valid video frames be... ,Will Replace with 0 to get V; Calculate the median Calculate the absolute deviation ; Calculate MAD value Mark outliers: ),otherwise =0; Then, collect the indices and values ​​of the non-outliers. For each outlier Find the nearest non-outlier values ​​on the left and right, using distance-weighted interpolation. , If no non-outlier values ​​can be interpolated, fill with the median of non-outlier values; the original list of boundary lines is then processed to form a new list of boundary lines. This eliminates the defect of unclear image boundaries caused by accidental reasons.

[0036] This invention uses a matching stitching algorithm to stitch and crop multiple frames of local images into a complete unfolded image of the blank's side surface, achieving the reconstruction of complete surface information. This solves the problem that a single view cannot cover the entire surface of a large workpiece, and also provides accurate data support for subsequent oxide scale identification, as detailed below: In S4, the method for stitching preprocessed video frames into a unfolded image of the billet's outer perimeter using boundary lines based on feature matching algorithms includes: Bottom cropping is performed sequentially on the preprocessed video frames. Horizontal cropping To obtain the effective width Where H is the frame height. This refers to the bottom cutting ratio. ; The SIFT detector is used to extract the previous frame of the preprocessed video frame. With the current frame Feature points , descriptor ; The SIFT detector is used to extract the previous frame of the preprocessed video frame. With the current frame Feature points , descriptor Using a brute-force matcher (BFMatcher) Perform K-nearest neighbor (KNN) matching to select a set of high-quality matching feature points. Where m is the nearest neighbor match and n is the second nearest neighbor match. For the closest distance, The second closest distance, ; If matching point 10. Calculate the homography matrix H using RANSAC robustness estimation, and then use the previous frame... Project the four corners onto the current frame and calculate the maximum projected x-coordinate. The previous frame before projection The overlap width at the four corners , if matching point Using a preset overlap width And by setting a preset overlap width limit, the overlap width after the limit is obtained by avoiding extreme values. ; Initialize a white background (255, 255, 255), and then use Gaussian weighted blending to obtain the stitched unfolded image. The average overlap width is obtained as Unfolded diagram with white background fill As the unfolded diagram of the outer perimeter of the billet, where I The corresponding image dimensions [rows, columns, channels], Gaussian weights , This is a weighted fusion function, where c is the column index within the overlapping region. This represents the starting x-coordinate of the current frame in the unfolded graph. The overlap width after limitation in the i-th video frame .

[0037] After the initial unfolding, this invention further stitches the left and right portions of the unfolded image together to compensate for the lack of perspective in unidirectional unfolding, achieving full circumferential coverage and thus obtaining a complete image of the product's circumference, as detailed below: In S4, the secondary stitching method for compensating for the lack of perspective in unidirectional development of the billet's outer peripheral surface development diagram based on NCC verification includes: like Figure 3 As shown, take the minimum height of the left and right portions of the unfolded outer circumference of the blank and cut it to a uniform height. ,in These represent the heights of the left and right sections, respectively. To perform the minimum operation; After unifying the height, the left and right portions of the unfolded outer perimeter of the billet are subjected to grayscale conversion and histogram equalization to obtain... , ,in , These are the left and right portions of the unfolded view of the outer perimeter of the billet after it has been standardized to a uniform height. This is a grayscale identifier. This is a color conversion function. For histogram equalization function, , These are grayscale images of the left and right portions of the image after conversion and equalization, respectively. Using SIFT estimates Calculate the candidate overlap width range ; NCC verification calculates the normalized cross-correlation coefficient. And select the one with the largest NCC value. As the optimal overlap width ,in For covariance, Standard deviation; Calculate the total width of the splicing Gaussian weighted fusion is applied to the overlapping areas, with the Gaussian weight of column c being... The corresponding columns of the left and right images are merged according to weights (1-α) and α to obtain the complete stitched image. As a complete unfolded view of the outer perimeter of the billet, in which , These represent the widths of the left and right portions of the unfolded diagram of the outer perimeter of the billet.

[0038] In S5, the method for identifying the pixel proportion of oxide scale flakes of different thicknesses on the surface of a billet in the unfolded image of the complete outer peripheral surface of the billet based on the brightness weighting method includes: Traverse the complete outer perimeter development of the billet column by column, exclude the continuous pure white (255,255,255) background, record the effective pixel range of each column, and count the total number of effective pixels; Based on the radiation characteristics of high-temperature metals, the brightness value of each effective pixel is calculated. ,in , , R, G, and B are the channel values, respectively. The brightness weighting method can be replaced by the CIELAB method (L =0.2126×R +0.7152×G +0.0722×B The accuracy will be similar after replacement, but the time consumption will increase.

[0039] The pixels are divided into multiple intervals based on the brightness value. Each interval corresponds to a different oxide thickness level. The number of pixels in each interval is counted and the proportion of the total effective pixels is calculated to obtain the thickness distribution ratio of the oxide scales.

[0040] This invention provides an application embodiment for testing the effectiveness of high-pressure water in removing oxide scale from product surfaces. The specific steps are as follows: S1: Data Acquisition: The detection system controls the camera to a preset position according to the product's specifications, ensuring the distance between the product and the camera is within the range of the depth of field and near depth of field, collecting qualified and clear data. The data acquisition cycle is determined by the product's rotation speed, and the surface video data of the product is acquired through the camera. The surface image of the product is obtained based on the product's rotation speed and the video frame extraction algorithm. S2: Data cleaning: To avoid interference from flames, glass dust, etc., bilateral filtering is used to remove interference, CLAHE balances the brightness unevenness caused by flames and enhances the details of areas covered by dust, Laplacian operator restores the blurred edges of dust, and SIFT+RANSAC feature matching is combined to filter out abnormal matches to ensure the stability of subsequent image stitching and other processing. S3: Data Feature Enhancement: The acquired product surface image data is enhanced by bilateral filtering denoising algorithm, CLAHE algorithm and Laplacian algorithm to avoid interference from factors such as complex lighting environment in the workshop and temperature differences between different products, so as to facilitate accurate identification of product boundaries in the subsequent process.

[0041] S4: Identify Product Boundaries: By fine-tuning the model through deep learning, product boundaries are identified. At the same time, outlier detection and interpolation replacement methods are used to handle the results of boundary identification anomalies.

[0042] S5: Photo stitching and cropping: such as Figure 4 As shown, based on preprocessed frames and boundary lines, the SIFT feature matching algorithm is used to stitch the photos together by setting an overlap threshold to achieve a planar unfolding of the product surface. Then, the heights of the left and right parts of the unfolded image are unified, the overlap width is checked using NCC, and finally, a Gaussian weighted fusion method is used to obtain a complete circumferential image of the product surface.

[0043] S6: Identification of the proportion of oxide scale: such as Figure 5 As shown, the complete circumferential image of the product surface is processed in grayscale, and the image is traversed column by column. The continuous pure white background is excluded, the effective pixel range of each column is recorded, and the total number of effective pixels is counted. Then, based on the radiation characteristics of high-temperature metals, weighted processing is adopted to avoid the deviation of simple grayscale values. The final proportion of oxide scales of different thicknesses is determined according to the final effect image.

[0044] like Figure 6 As shown, this invention provides a deep learning-based system for identifying the percentage of oxide scale on the surface of billets, applied to a deep learning-based method for identifying the percentage of oxide scale on the surface of billets. The system includes: The data acquisition unit is used to control the camera to adaptively adjust the shooting position according to the specifications of the billet product, and to acquire video of the outer circumference surface of the billet by adapting to the billet rotation speed. The preprocessing unit is used to extract multiple valid video frames from the video of the outer surface and perform bilateral filtering noise reduction, CLAHE contrast enhancement and Laplacian sharpening in sequence to obtain preprocessed video frames. The boundary recognition unit is used to identify the blank boundary line in the preprocessed video frame based on a deep learning model, and to perform anomaly detection and interpolation correction on the boundary line recognition results. The image stitching unit is used to stitch preprocessed video frames into a billet outer peripheral surface unfolded image based on the feature matching algorithm using boundary lines, and to perform secondary stitching on the billet outer peripheral surface unfolded image based on NCC verification to make up for the lack of viewpoints in unidirectional unfolding, so as to obtain a complete billet outer peripheral surface unfolded image. The proportion analysis unit identifies the pixel proportion of oxide scale flakes of different thicknesses on the surface of the billet in the unfolded image of the complete outer peripheral surface of the billet based on the brightness weighting method.

[0045] The present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement a method for identifying the percentage of oxide scale on the surface of a billet based on deep learning.

[0046] This invention uses camera position control and rotation speed adaptation acquisition logic to cover cylindrical blanks of different diameters and lengths, eliminating the need for manual adjustment of shooting parameters, adapting to the dynamic production rhythm of the workshop, and reducing operating costs.

[0047] This invention addresses interference from workshop flames, glass dust, and uneven brightness by employing a combination of bilateral filtering, CLAHE contrast enhancement, and Laplacian sharpening preprocessing to effectively remove noise and enhance oxide scale texture details, thereby improving the signal-to-noise ratio of the preprocessed image.

[0048] This invention uses "deep learning fine-tuning model dynamic boundary detection" to obtain product boundaries, and uses "matching and stitching algorithm" to stitch and crop multiple frames of local images into a complete unfolded view of the blank side, realizing the reconstruction of complete surface information, solving the defect that a single view cannot cover the entire surface of a large workpiece, and providing accurate data support for subsequent oxide scale identification.

[0049] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A method for identifying the percentage of oxide scale on the surface of a billet based on deep learning, characterized in that, Includes the following steps: S1: Based on the specifications of the billet product, control the camera to adaptively adjust the shooting position and collect video of the outer circumference surface of the billet around the entire circumference, adapting to the billet rotation speed. S2: Extract multiple valid video frames from the video of the outer peripheral surface, and perform preprocessing such as bilateral filtering for noise reduction, CLAHE contrast enhancement and Laplacian sharpening in sequence to obtain preprocessed video frames; S3: Based on a deep learning model, identify the blank boundary line in the preprocessed video frame, and perform anomaly detection and interpolation correction on the boundary line identification results; S4: Based on the feature matching algorithm, the preprocessed video frames are stitched together to form the unfolded image of the outer periphery of the billet. The unfolded image is then cropped using the boundary lines to eliminate irrelevant backgrounds. Based on NCC verification, the unfolded image of the outer periphery of the billet is stitched together again to compensate for the lack of viewpoints in unidirectional unfolding, thus obtaining the complete unfolded image of the outer periphery of the billet. S5: Identify the pixel ratio of oxide scale flakes of different thicknesses on the surface of the billet in the unfolded image of the complete outer peripheral surface of the billet based on the brightness weighting method.

2. The method for identifying the proportion of oxide scale on the surface of a billet based on deep learning according to claim 1, characterized in that: In step S2, the method for extracting multiple valid video frames from the video of the outer peripheral surface includes: Select the start extraction time Frame interval k, number of target extraction frames N, rotation angle The starting frame index is calculated by combining the video frame rate (fps). With the set of frame indices to be extracted ,in For floor operations, if If so, it will be automatically truncated to the total number of frames; In different shooting scenarios, through rotation matrix Translation vector Rotate all frames to be extracted to a uniform orientation, and then determine the pixel coordinates of the video frames. The coordinate transformation relationship with the original coordinates (x, y) is as follows: in It consists of the original frame width W and the original frame height H of the video frame; In the set of frame indexes to be extracted Extract M video frames to form a list of valid video frames .

3. The method for identifying the proportion of oxide scale on the surface of a billet based on deep learning according to claim 2, characterized in that: In S2, the preprocessing methods for bilateral filtering denoising, CLAHE contrast enhancement, and Laplacian sharpening include: Set the filter diameter d and the color standard deviation Spatial standard deviation Set to noise reduction intensity Linear correlation involves performing bilateral filtering on valid video frames, resulting in the effective video frame pixels after bilateral filtering. in For the neighborhood, These are Gaussian functions for the color domain and the spatial domain, respectively. The normalization coefficient is... Center pixel pixel intensity, For the neighborhood medium pixel Pixel intensity; The effective video frames after bilateral filtering are converted to the YCrCb color space, the luminance channel Y is extracted, and the contrast is enhanced using the CLAHE algorithm to obtain the enhanced luminance channel. Then, the Cr and Cb channels are merged and converted back to the BGR color space to obtain the effective video frames after CLAHE contrast enhancement. Due to cropping limitations, Determine the number of grid divisions; By comparing with sharpening intensity Associated convolutional kernels Effective video frames after CLAHE contrast enhancement Perform convolution enhancement to obtain sharpened pixel values. And sharpened pixel values After truncation and limiting the value to the range of 0-255, the effective video frames after Laplacian sharpening are obtained. As preprocessed video frames, For convolution operations, This is a truncation function.

4. The method for identifying the proportion of oxide scale on the surface of a billet based on deep learning according to claim 3, characterized in that: In step S3, the method for identifying billet boundary lines in preprocessed video frames based on a deep learning model, and performing anomaly detection and interpolation correction on the boundary line identification results, includes: The preprocessed video frames are normalized, channel-converted, and pixel-value-standardized according to a preset size to obtain a tensor of the image to be detected that is adapted to the input of the deep learning model. The image tensor is input into a billet boundary line detection model pre-trained based on the YOLOv12 single-stage object detection algorithm. The training process of this boundary line detection model is as follows: using video frame images with labeled billet boundary line regions as training samples, a billet boundary line detection dataset containing training, validation, and test sets is constructed. Feature extraction is performed based on the backbone network of the YOLOv12 algorithm. Multi-scale feature maps are fused through the SPPF spatial pyramid pooling structure, which serves as the neck network. Then, the detection head network performs bounding box regression and class confidence prediction on the fused feature maps. The CIoU loss function is used to optimize the bounding box regression accuracy, and the cross-entropy loss function is used to optimize the class classification accuracy. After multiple rounds of iterative training and validation set tuning, a convergent boundary line detection model with detection accuracy meeting the preset threshold is obtained. In the inference phase, this boundary line detection model performs feature extraction and target detection on the input tensor of the image to be detected, and outputs predicted bounding box parameters representing the position of the billet boundary line, denoted as the bounding box. ,in The coordinates of the top left corner of the bounding box. The coordinates are the lower right corner of the bounding box; at the same time, the boundary line detection model will also output the detection confidence of the blank boundary line corresponding to the bounding box, which is used for subsequent screening of valid bounding boxes; For the detection results of a single frame of a valid video frame, select the bounding box with the highest confidence. Take the bounding box The average of the upper and lower y-coordinates is used as the upper boundary line. ,in , Boundary frames Upper and lower y-coordinates; Summarize the boundary lines of all valid video frames, and fill the boundary lines of valid video frames that failed bounding box detection with 0s to obtain a boundary line list. ,in , These are the boundary lines of the 1st and Mth valid video frames, respectively; Calculate the median separately absolute deviation MAD value ,in Let i be the boundary line of the i-th valid video frame. , and represent the absolute deviations of the i-th and M-th valid video frames, respectively. For the median operator, This is the absolute value operator; Will corresponding Mark outliers as and the remaining corresponding Mark non-outlier values ​​as , Collect the indices and values ​​of non-outliers And for each outlier Find the nearest non-outlier values ​​on the left and right, and then check the outlier values. Based on distance-weighted interpolation processing ,in , These are outliers The nearest non-outlier values ​​on the left and right sides , They are respectively , The index is filled with the median of non-outliers when no interpolation is available; List of dividing lines After interpolation, it is converted into a new list of boundary lines. .

5. The method for identifying the proportion of oxide scale on the surface of a billet based on deep learning according to claim 4, characterized in that: In step S4, the method for stitching preprocessed video frames into a unfolded diagram of the outer periphery of a billet using a feature matching algorithm based on boundary lines includes: Bottom cropping is performed sequentially on the preprocessed video frames. Horizontal cropping To obtain the effective width Where H is the frame height. This refers to the bottom cutting ratio. ; The SIFT detector is used to extract the previous frame of the preprocessed video frame. With the current frame Feature points , descriptor ; Descriptors are matched using the brute-force matcher BFMatcher. Perform K-nearest neighbor (KNN) matching to select a set of high-quality matching feature points. Where m is the nearest neighbor match and n is the second nearest neighbor match. For the closest distance, The second closest distance, ; If matching point ≥10, calculate the homography matrix H using RANSAC robust estimation, and then use the previous frame Projecting the four corners onto the current frame, calculating the maximum projected x-coordinate. The previous frame before projection The overlap width at the four corners , if matching point Using a preset overlap width And by setting a preset overlap width limit, the overlap width after the limit is obtained by avoiding extreme values. ; Initialize a white background (255, 255, 255), and then use Gaussian weighted blending to obtain the stitched unfolded image. The average overlap width is obtained as Unfolded diagram with white background fill As the unfolded diagram of the outer perimeter of the billet, where I The corresponding image dimensions [rows, columns, channels], Gaussian weights , This is a weighted fusion function, where c is the column index within the overlapping region. This represents the starting x-coordinate of the current frame in the unfolded graph. The overlap width after limitation in the i-th video frame .

6. The method for identifying the proportion of oxide scale on the surface of a billet based on deep learning according to claim 5, characterized in that: In step S4, the method for secondary splicing of the developed view of the outer periphery of the billet based on NCC verification to compensate for the lack of perspective in unidirectional development includes: Take the minimum height of the left and right portions of the unfolded outer circumference of the blank, and cut it to a uniform height. ,in These represent the heights of the left and right sections, respectively. To perform the minimum operation; After unifying the height, the left and right portions of the unfolded outer perimeter of the billet are subjected to grayscale conversion and histogram equalization to obtain... , ,in , These are the left and right portions of the unfolded view of the outer perimeter of the billet after it has been standardized to a uniform height. This is a grayscale identifier. This is a color conversion function. For histogram equalization function, , These are grayscale images of the left and right portions of the image after conversion and equalization, respectively. Using SIFT estimates Calculate the candidate overlap width range ; NCC verification calculates the normalized cross-correlation coefficient. And select the one with the largest NCC value. As the optimal overlap width ,in For covariance, Standard deviation; Calculate the total width of the splicing Gaussian weighted fusion is applied to the overlapping areas, with the Gaussian weight of column c being... The corresponding columns of the left and right images are merged according to weights (1-α) and α to obtain the complete stitched image. As a complete unfolded view of the outer perimeter of the billet, in which , These represent the widths of the left and right portions of the unfolded diagram of the outer perimeter of the billet.

7. The method for identifying the proportion of oxide scale on the surface of a billet based on deep learning according to claim 6, characterized in that: In step S5, the method for identifying the pixel proportion of oxide scale flakes of different thicknesses on the surface of the billet in the unfolded image of the complete outer peripheral surface of the billet based on the brightness weighting method includes: Traverse the complete outer perimeter development of the billet column by column, exclude the continuous pure white (255,255,255) background, record the effective pixel range of each column, and count the total number of effective pixels; Based on the radiation characteristics of high-temperature metals, the brightness value of each effective pixel is calculated. ,in , , R, G, and B are the channel values, respectively. The pixels are divided into multiple intervals based on the brightness value. Each interval corresponds to a different oxide thickness level. The number of pixels in each interval is counted and the proportion of the total effective pixels is calculated to obtain the thickness distribution ratio of the oxide scales.

8. A deep learning-based system for identifying the percentage of oxide scale on the surface of billets, characterized in that, The system, which is applied to the deep learning-based method for identifying the oxide scale ratio on the surface of a billet as described in any one of claims 1-7, comprises: The data acquisition unit is used to control the camera to adaptively adjust the shooting position according to the specifications of the billet product, and to acquire video of the outer circumference surface of the billet by adapting to the billet rotation speed. The preprocessing unit is used to extract multiple valid video frames from the video of the outer peripheral surface, and sequentially perform bilateral filtering noise reduction, CLAHE contrast enhancement and Laplacian sharpening preprocessing to obtain preprocessed video frames; The boundary recognition unit is used to identify the blank boundary line in the preprocessed video frame based on a deep learning model, and to perform anomaly detection and interpolation correction on the boundary line recognition results. The image stitching unit is used to stitch preprocessed video frames into a billet outer peripheral surface unfolded image based on the feature matching algorithm using boundary lines, and to perform secondary stitching on the billet outer peripheral surface unfolded image based on NCC verification to make up for the lack of viewpoints in unidirectional unfolding, so as to obtain a complete billet outer peripheral surface unfolded image. The proportion analysis unit identifies the pixel proportion of oxide scale flakes of different thicknesses on the surface of the billet in the unfolded image of the complete outer peripheral surface of the billet based on the brightness weighting method.

Images