Metal ingot surface defect detection method, device, equipment and readable storage medium

Abstract

This invention discloses a method, apparatus, device, and readable storage medium for detecting surface defects in metal ingots, relating to the field of defect detection technology. The method includes: acquiring an RGB image and a 3D point cloud of the metal surface of a metal ingot; obtaining a target curvature map from the 3D point cloud; training a geometric perception network (GPN) based on the RGB image and the target curvature map to obtain a trained GPN; training an initial teacher model based on the RGB image, the target curvature map, and real defect annotations to obtain a target teacher model; inputting a preset RGB image of the metal surface into the target teacher model to obtain soft labels and their corresponding intermediate features; training a student model based on the preset RGB image, soft labels, intermediate features, and real defect annotations to obtain a trained student model; and performing surface defect detection on the metal ingot using the trained student model. This achieves reliable detection of surface defects in metal ingots.

Description

Technical Field

[0001] This invention relates to the field of defect detection, and more particularly to a method, apparatus, equipment, and readable storage medium for detecting defects on the surface of metal ingots. Background Technology

[0002] In the smelting and production sector, the casting process for lead, zinc, and their alloys is a crucial step. This involves melting the metal, pouring it into a mold, and allowing it to cool and solidify, thus obtaining metal ingots of specific shapes and sizes. Qualified ingots must undergo a series of processes, including stacking, packaging, weighing, and labeling, before being stored and moved to the warehouse. Quality control of the cast ingots is extremely strict throughout the casting process. Any ingots with surface defects such as slag, burrs, protrusions, or dripping must be identified and removed promptly; otherwise, it will affect the quality of downstream production.

[0003] Currently, waste ingot detection mainly relies on manual visual inspection and manual rejection. However, this method is difficult to adapt to the continuous and high-speed production rhythm, especially in harsh working environments with high temperatures and dust, where operators experience high labor intensity and are prone to fatigue, leading to a significant increase in the rate of missed detections. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to overcome the shortcomings of the prior art and provide a method, apparatus, device and readable storage medium for detecting surface defects of metal ingots.

[0005] This invention provides the following technical solution: In a first aspect, the present invention provides a method for detecting surface defects in a metal ingot, the method comprising: Acquire RGB images and 3D point clouds of the metal surface of a metal ingot, and obtain a target curvature map of the RGB image of the metal surface based on the 3D point cloud; The RGB image of the metal surface is used as input, and the target curvature map is used as a label to train the geometry perception network, resulting in a trained geometry perception network. The trained geometry perception network includes a first candidate RGB encoder and a first candidate curvature prediction head. The RGB image of the metal surface and the target curvature map are used as the main inputs, and the real defect annotations corresponding to the RGB image of the metal surface are used as the main labels. The RGB image of the metal surface is used as the secondary input, and the target curvature map is used as the secondary label to train the initial teacher model and obtain the target teacher model. The initial teacher model includes the first candidate RGB encoder, the curvature encoder, the curvature attention module, the first defect detection head, and the second candidate curvature prediction head. A preset RGB image of a metal surface is input into the target teacher model to obtain soft labels and their corresponding intermediate features. The target teacher model includes a trained first candidate RGB encoder, a trained curvature encoder, a trained curvature attention module, and a trained first defect detection head. The preset metal surface RGB image is used as input, the soft label and the real defect annotation corresponding to the preset metal surface RGB image are used as labels, and the intermediate feature corresponding to the soft label is used as supervision signal to train the student model and obtain the trained student model. The student model includes a second candidate RGB encoder and a second defect detection head. Surface defect detection of metal ingots is performed based on the trained student model.

[0006] In an optional implementation, obtaining the target curvature map of the metal surface RGB image based on the three-dimensional point cloud includes: Multi-scale normal vector estimation is performed on the three-dimensional point cloud to obtain the denoised normal vector; The Gaussian curvature and average curvature are calculated based on the denoised normal vector, and defect-sensitive curvature encoding is performed based on the Gaussian curvature and average curvature to obtain the defect-sensitive curvature. The defect-sensitive curvature is aligned with RGB edges based on the RGB image of the metal surface to obtain the target curvature map.

[0007] In an optional implementation, the step of encoding defect-sensitive curvature based on the Gaussian curvature and the average curvature to obtain the defect-sensitive curvature includes: Obtain the absolute value of the Gaussian curvature, and transform the average curvature based on the activation function; The defect-sensitive curvature is obtained by multiplying the absolute value of the Gaussian curvature by the transformed average curvature.

[0008] In an optional implementation, the step of using the RGB image of the metal surface and the target curvature map as the main input, the real defect annotation corresponding to the RGB image of the metal surface as the main label, and the RGB image of the metal surface as the secondary input and the target curvature map as the secondary label to train the initial teacher model and obtain the target teacher model includes: The first candidate RGB encoder obtains RGB features based on the RGB image of the metal surface, and the curvature encoder obtains curvature features based on the target curvature map. The RGB features and the curvature features are fused using the curvature attention module to obtain the target fused features; The first defect detection head obtains the defect prediction result based on the target fusion features, and the second candidate curvature prediction head obtains the curvature prediction result based on the RGB features; A first loss value is calculated based on the defect prediction result and the actual defect annotation, and a second loss value is calculated based on the curvature prediction result and the target curvature map. The target loss value is obtained by weighted summation of the first loss value and the second loss value, and the network parameters of the initial teacher model are optimized based on the target loss value.

[0009] In an optional implementation, the step of fusing the RGB features and the curvature features through the curvature attention module to obtain the target fused features includes: The curvature confidence level is calculated based on the curvature features using the curvature attention module. Align the curvature feature with the RGB feature, and obtain the modulated curvature feature based on the aligned curvature feature and the curvature confidence. An initial fused feature is obtained by fusing the RGB features and the modulated curvature features based on an attention mechanism; The initial fusion feature is residually connected based on the RGB features to obtain the target fusion feature.

[0010] In an optional implementation, obtaining the modulated curvature features based on the aligned curvature features and the curvature confidence includes: The aligned curvature feature is multiplied element-wise with the curvature confidence to obtain the modulated curvature feature.

[0011] In an optional implementation, the step of detecting surface defects on metal ingots based on the trained student model includes: The RGB image of the metal surface of the metal ingot to be tested is input into the trained student model to obtain the surface defect detection result of the metal ingot based on the RGB image of the metal surface to be tested.

[0012] In a second aspect, the present invention provides a device for detecting surface defects in metal ingots, the device comprising: The data processing module is used to acquire RGB images and three-dimensional point clouds of the metal surface of a metal ingot, and to obtain a target curvature map of the RGB image of the metal surface based on the three-dimensional point cloud. The first training module is used to train a geometry perception network by taking the RGB image of the metal surface as input and the target curvature map as a label, and to obtain a trained geometry perception network. The trained geometry perception network includes a first candidate RGB encoder and a first candidate curvature prediction head. The second training module is used to take the RGB image of the metal surface and the target curvature map as the main input, the real defect annotation corresponding to the RGB image of the metal surface as the main label, the RGB image of the metal surface as the secondary input, and the target curvature map as the secondary label, to train the initial teacher model and obtain the target teacher model. The initial teacher model includes the first candidate RGB encoder, the curvature encoder, the curvature attention module, the first defect detection head and the second candidate curvature prediction head. The third training module is used to input a preset RGB image of a metal surface into the target teacher model to obtain soft labels and their corresponding intermediate features. The target teacher model includes a trained first candidate RGB encoder, a trained curvature encoder, a trained curvature attention module, and a trained first defect detection head. The fourth training module is used to take the preset metal surface RGB image as input, the soft label and the real defect annotation corresponding to the preset metal surface RGB image as labels, and the intermediate features corresponding to the soft label as supervision signals to train the student model and obtain the trained student model. The student model includes a second candidate RGB encoder and a second defect detection head. The defect detection module is used to detect surface defects of metal ingots based on the trained student model.

[0013] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the metal ingot surface defect detection method as described in any of the foregoing embodiments.

[0014] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for detecting surface defects of metal ingots as described in any of the foregoing embodiments.

[0015] The present invention discloses a method, apparatus, device, and readable storage medium for detecting surface defects of metal ingots. By fusing RGB features in the RGB image of the metal surface with three-dimensional geometric curvature features in the target curvature map, the detection rate of small or low-contrast defects such as slag and burrs is improved. By adopting geometric perception pre-training and a teacher-student knowledge distillation two-stage strategy, the number of model parameters is compressed and the inference speed is improved while ensuring detection accuracy. This enables efficient and accurate defect detection on edge devices and meets the needs of real-time industrial detection. Attached Figure Description

[0016] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope of protection of the present invention. In the various drawings, similar components are numbered similarly.

[0017] Figure 1 A flowchart of the metal ingot surface defect detection method proposed in this embodiment is shown; Figure 2 Another flowchart of the metal ingot surface defect detection method proposed in this embodiment is shown; Figure 3 This diagram illustrates another step in the process of the metal ingot surface defect detection method proposed in this embodiment. Figure 4 This diagram illustrates another step in the process of the metal ingot surface defect detection method proposed in this embodiment. Figure 5 A schematic diagram of the metal ingot surface defect detection device proposed in this embodiment is shown.

[0018] Explanation of reference numerals in the attached diagram: 500 - Metal ingot surface defect detection device; 501 - Data processing module; 502 - First training module; 503 - Second training module; 504 - Third training module; 505 - Fourth training module; 506 - Defect detection module. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0021] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0022] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0023] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0024] Example 1 This disclosure provides a method for detecting surface defects in metal ingots.

[0025] Please see Figure 1 The method for detecting surface defects in metal ingots includes steps S101 to S106, each of which will be described in detail below.

[0026] Step S101: Obtain the RGB image and three-dimensional point cloud of the metal surface of the metal ingot, and obtain the target curvature map of the RGB image of the metal surface based on the three-dimensional point cloud.

[0027] In this embodiment, a color RGB camera combined with a TOF 3D depth camera is used to acquire RGB images and 3D point clouds of the metal surface of the metal ingot, and the target curvature map of the RGB image of the metal surface is further obtained based on the 3D point cloud.

[0028] Please see Figure 2 In one specific embodiment, step S101 includes steps S1011 to S1013, and each step is described in detail below.

[0029] Step S1011: Perform multi-scale normal vector estimation on the three-dimensional point cloud to obtain the denoised normal vector.

[0030] In this embodiment, multi-scale normal vector estimation is performed on the original 3D point cloud to obtain stable, non-drifting, and anti-reflective normal vectors after noise reduction, thereby reducing the interference of reflective noise on the true curvature. For example, multiple sets of normal vectors are calculated using three neighborhood radii of 2mm, 5mm, and 10mm, and then fused into a single set of noise-reduced normal vectors.

[0031] Step S1012: Calculate the Gaussian curvature and average curvature based on the denoised normal vector, and perform defect-sensitive curvature encoding based on the Gaussian curvature and average curvature to obtain the defect-sensitive curvature.

[0032] In this embodiment, the maximum and minimum normal curvatures k1 and k2 at a given point on the surface are determined based on the denoised normal vector, and the Gaussian curvature K = k1 × k2 and the Gaussian curvature H = 1 / 2 × (k1 + k2).

[0033] Furthermore, defects are usually manifested as regions of abrupt curvature change. Therefore, defect-sensitive curvature encoding is performed on Gaussian curvature and average curvature to obtain defect-sensitive curvature.

[0034] In one specific embodiment, step S1012 includes: obtaining the absolute value of the Gaussian curvature, transforming the average curvature based on an activation function; multiplying the absolute value of the Gaussian curvature by the transformed average curvature to obtain the defect-sensitive curvature.

[0035] In this embodiment, the absolute value of the Gaussian curvature is obtained, thereby turning all abrupt changes in concavity and convexity into large values ​​and normal surfaces into small values; the average curvature is transformed based on the activation function, thereby compressing the average curvature to [0,1] and stabilizing the numerical range.

[0036] Furthermore, the absolute value of the Gaussian curvature is multiplied by the transformed average curvature. The value is large only where the curvature changes drastically, thus obtaining the defect-sensitive curvature, i.e. the defect highlight map, which transforms the true curvature into a feature that highlights defects and suppresses normal areas.

[0037] Step S1013: Perform RGB edge alignment on the defect-sensitive curvature based on the RGB image of the metal surface to obtain the target curvature map.

[0038] In this embodiment, Canny edge detection is performed on the defect-sensitive curvature based on the RGB image of the metal surface, and the curvature weight of the edge position is enhanced to achieve RGB edge alignment. Finally, it is normalized to 0~1 to obtain the target curvature map.

[0039] Understandably, by aligning with the RGB edges and filtering out false curvature abrupt changes, a two-dimensional curvature feature map with spatial alignment, strong defect response, and low noise is ultimately generated, which is the target curvature map.

[0040] Step S102: The RGB image of the metal surface is used as input and the target curvature map is used as a label to train the geometry perception network, resulting in a trained geometry perception network. The trained geometry perception network includes a first candidate RGB encoder and a first candidate curvature prediction head.

[0041] In this embodiment, the RGB image of the metal surface is used as input and the target curvature map is used as a label to train the geometry perception network, thereby obtaining a first candidate RGB encoder that can be used to implicitly infer RGB features containing 3D geometric features from the 2D RGB image of the metal surface, and a first candidate curvature prediction head for inferring curvature based on the RGB features.

[0042] Understandably, employing a self-supervised learning approach, using the target curvature map as a supervisory signal, endows the first candidate RGB encoder with preliminary geometric reasoning capabilities. This method enhances the model's sensitivity to 3D deformation, enabling the RGB encoder, which originally lacked depth perception capabilities, to perform better in subsequent tasks.

[0043] It should be noted that the curvature prediction loss The calculation formula is: , For predicted values, is the target value, and N is the number of parameters to be aligned.

[0044] Step S103: The RGB image of the metal surface and the target curvature map are used as the main inputs, the real defect annotations corresponding to the RGB image of the metal surface are used as the main labels, the RGB image of the metal surface is used as the secondary inputs, and the target curvature map is used as the secondary labels, to train the initial teacher model and obtain the target teacher model. The initial teacher model includes the first candidate RGB encoder, the curvature encoder, the curvature attention module, the first defect detection head and the second candidate curvature prediction head.

[0045] In this embodiment, the RGB image of the metal surface and the target curvature map are used as the main inputs, and the real defect annotations corresponding to the RGB image of the metal surface are used as the main labels. At the same time, the RGB image of the metal surface is used as the secondary input, and the target curvature map is used as the secondary label. These are used as regularization terms to perform dual-task collaborative training of defect detection and curvature prediction on the initial teacher model, so as to obtain the target teacher model and continuously enhance the geometric consistency of the model in multimodal feature learning.

[0046] The initial teacher model includes a first candidate RGB encoder, a curvature encoder, a curvature attention module, a first defect detection head, and a second candidate curvature prediction head. By reusing the first candidate RGB encoder pre-trained in step S102, convergence is accelerated and the upper limit of the teacher model is improved.

[0047] Please see Figure 3 In one specific embodiment, step S103 includes steps S1031 to S1035, and each step is described in detail below.

[0048] Step S1031: Obtain RGB features from the RGB image of the metal surface using the first candidate RGB encoder, and obtain curvature features from the target curvature map using the curvature encoder.

[0049] In this embodiment, the first candidate RGB encoder extracts RGB features from the RGB image of the metal surface, and the curvature encoder extracts curvature features from the target curvature map. The first candidate RGB encoder can be Yolov8l.

[0050] Step S1032: The RGB features and the curvature features are fused by the curvature attention module to obtain the target fused features.

[0051] In this embodiment, the curvature attention module fuses the RGB features and the curvature features to ensure that the target fused features accurately inject defect-sensitive geometric clues while maintaining the RGB texture details, thus solving the dilemma of curvature noise overwhelming defect textures or RGB noise interfering with geometric judgment.

[0052] Please see Figure 4 In one specific embodiment, step S1032 includes steps S401 to S404, and each step is described in detail below.

[0053] Step S401: The curvature confidence is calculated based on the curvature features by the curvature attention module.

[0054] In this embodiment, the curvature attention module calculates curvature confidence based on curvature features, representing the reliability of the curvature features at that location. For example, curvature confidence Conf = σ(Conv(F_curv)), where F_curv is a curvature feature map with shape [B, 1, H, W], Conv is a 1×1 convolutional layer, and σ is a Sigmoid activation function.

[0055] Understandably, the curvature confidence score is calculated at each location, and this confidence score is used to modulate the feature weights. This confidence-guided mechanism effectively overcomes the interference of reflective surfaces on RGB images, ensuring that the model can rely more on stable curvature geometry features under poor lighting conditions.

[0056] Step S402: Align the curvature feature with the RGB feature, and obtain the modulated curvature feature based on the aligned curvature feature and the curvature confidence.

[0057] In this embodiment, the curvature features are spatially aligned with the RGB features, and the modulated curvature features are obtained based on the aligned curvature features and the curvature confidence. For example, the aligned curvature features are multiplied element-wise with the curvature confidence to obtain the modulated curvature features, thereby weighting each channel and each position of the curvature feature map once. Features in high-confidence regions are preserved or even enhanced, while features in low-confidence regions are suppressed.

[0058] Step S403: Based on the attention mechanism, the RGB features and the modulated curvature features are fused to obtain the initial fused features.

[0059] In this embodiment, the initial fused features are obtained by fusing the RGB features of color and the modulated curvature features based on the attention mechanism.

[0060] Step S404: Perform residual connection on the initial fusion feature based on the RGB feature to obtain the target fusion feature.

[0061] In this embodiment, residual connections are performed on the initial fusion features based on the RGB features. The RGB features are the main body, and geometric information dynamically extracted from the curvature features and weighted by confidence is injected to obtain the target fusion feature F_fused, which is expressed as: F_fused=F_rgb+α×Attn, where F_rgb is the RGB feature, α is a learnable scaling parameter, and Attn is the initial fusion feature.

[0062] Step S1033: Obtain the defect prediction result by the first defect detection head based on the target fusion features, and obtain the curvature prediction result by the second candidate curvature prediction head based on the RGB features.

[0063] In this embodiment, the first defect detection head infers the defect prediction result based on the target fusion features, and the second candidate curvature prediction head infers the curvature prediction result based on the RGB features.

[0064] Step S1034: Calculate a first loss value based on the defect prediction result and the actual defect annotation, and calculate a second loss value based on the curvature prediction result and the target curvature map.

[0065] In this embodiment, the first loss value is calculated based on the defect prediction results and the actual defect annotations. First loss value It is the weighted sum of classification loss, location loss, and confidence loss.

[0066] Simultaneously, a second loss value, namely the curvature prediction loss, is calculated based on the curvature prediction results and the target curvature map. .

[0067] Step S1035: Perform a weighted summation of the first loss value and the second loss value to obtain the target loss value, and optimize the network parameters of the initial teacher model based on the target loss value.

[0068] In this embodiment, the first loss value is determined according to predefined parameters. Second loss value We perform weighted summation to obtain the target loss value, and then optimize the network parameters of the initial teacher model based on the target loss value.

[0069] Step S104: Input the preset RGB image of the metal surface into the target teacher model to obtain soft labels and their corresponding intermediate features. The target teacher model includes a trained first candidate RGB encoder, a trained curvature encoder, a trained curvature attention module, and a trained first defect detection head.

[0070] In this embodiment, a preset RGB image of a metal surface is input into the target teacher model to obtain soft labels and their corresponding intermediate features, which serve as a carrier of the teacher's high-precision capabilities. The soft labels (such as defect probability distributions and location heatmaps) contain the teacher's comprehensive judgment on the existence, category, boundary ambiguity, and overlapping relationships of multiple defects, far exceeding the information content of binary annotations. The intermediate features (such as attention weight maps and multi-layer feature maps) reflect how the teacher thinks, providing the student model with a transferable cognitive process, rather than merely imitating the result.

[0071] Understandably, if the second candidate curvature prediction head is removed from the initial teacher model after training, the target teacher model includes the first candidate RGB encoder after training, the curvature encoder after training, the curvature attention module after training, and the first defect detection head after training.

[0072] Step S105: The preset metal surface RGB image is used as input, the soft label and the real defect annotation corresponding to the preset metal surface RGB image are used as labels, and the intermediate features corresponding to the soft label are used as supervision signals to train the student model and obtain the trained student model. The student model includes a second candidate RGB encoder and a second defect detection head.

[0073] In this embodiment, a preset RGB image of a metal surface is used as input, and soft labels and corresponding real defect annotations are used as labels. The real defect annotations are hard labels, and the intermediate features corresponding to the soft labels are used as supervision signals to train a student model, resulting in a trained student model. The student model includes a second candidate RGB encoder and a second defect detection head. The second candidate RGB encoder can be Yolov8n, and its parameter count is smaller than that of the first candidate RGB encoder.

[0074] It should be noted that the loss function for the student model is: In the formula, The loss value for the student model. , , For predefined parameters, For the student model's defect prediction values, Labeling for actual defects. Cross-entropy loss is used to constrain the accuracy of hard-label classification. For soft label loss, KL divergence is used to measure the output probability of the student model. With the teacher model output probability The distributional differences are expressed as follows: ; The feature layer loss is used to align the feature map of the third layer of the teacher model with the MSE loss. Feature map of the corresponding layer of the student model Spatial distribution representation: .

[0075] Understandably, knowledge distillation transfers knowledge from the teacher network to the lightweight student network, and its loss function integrates supervision from real labels, guidance from teacher soft labels, and imitation of intermediate features.

[0076] Step S106: Detect surface defects of metal ingots based on the trained student model.

[0077] In this embodiment, the RGB image of the metal surface of the metal ingot to be tested is input into the trained student model to obtain the surface defect detection result of the metal ingot based on the RGB image of the metal surface to be tested.

[0078] The metal ingot surface defect detection method proposed in this embodiment improves the detection rate of small or low-contrast defects such as slag and burrs by fusing RGB features in the RGB image of the metal surface with three-dimensional geometric curvature features in the target curvature map. It adopts a geometric perception pre-training and a teacher-student knowledge distillation two-stage strategy to compress the number of model parameters and improve the inference speed while ensuring detection accuracy.

[0079] Example 2 Furthermore, this disclosure provides a metal ingot surface defect detection device 500, please refer to [link to relevant documentation]. Figure 5 The device includes: Data processing module 501 is used to acquire RGB images and three-dimensional point clouds of the metal surface of a metal ingot, and to obtain a target curvature map of the RGB image of the metal surface based on the three-dimensional point cloud. The first training module 502 is used to train a geometry perception network by taking the RGB image of the metal surface as input and the target curvature map as a label, and to obtain a trained geometry perception network. The trained geometry perception network includes a first candidate RGB encoder and a first candidate curvature prediction head. The second training module 503 is used to take the RGB image of the metal surface and the target curvature map as the main input, the real defect annotation corresponding to the RGB image of the metal surface as the main label, the RGB image of the metal surface as the secondary input, and the target curvature map as the secondary label, to train the initial teacher model and obtain the target teacher model. The initial teacher model includes the first candidate RGB encoder, the curvature encoder, the curvature attention module, the first defect detection head and the second candidate curvature prediction head. The third training module 504 is used to input a preset RGB image of a metal surface into the target teacher model to obtain soft labels and their corresponding intermediate features. The target teacher model includes a trained first candidate RGB encoder, a trained curvature encoder, a trained curvature attention module, and a trained first defect detection head. The fourth training module 505 is used to take the preset metal surface RGB image as input, the soft label and the real defect annotation corresponding to the preset metal surface RGB image as labels, and the intermediate feature corresponding to the soft label as supervision signal to train the student model and obtain the trained student model. The student model includes a second candidate RGB encoder and a second defect detection head. The defect detection module 506 is used to detect surface defects of metal ingots based on the trained student model.

[0080] In an optional implementation, the data processing module 501 is further configured to perform multi-scale normal vector estimation on the three-dimensional point cloud to obtain a denoised normal vector; calculate Gaussian curvature and average curvature based on the denoised normal vector; perform defect-sensitive curvature encoding based on the Gaussian curvature and average curvature to obtain a defect-sensitive curvature; and perform RGB edge alignment on the defect-sensitive curvature based on the RGB image of the metal surface to obtain the target curvature map.

[0081] In an optional implementation, the data processing module 501 is further configured to obtain the absolute value of the Gaussian curvature, transform the average curvature based on an activation function, and multiply the absolute value of the Gaussian curvature by the transformed average curvature to obtain the defect-sensitive curvature.

[0082] In an optional implementation, the second training module 503 is further configured to: acquire RGB features from the RGB image of the metal surface using the first candidate RGB encoder; acquire curvature features from the target curvature map using the curvature encoder; fuse the RGB features and the curvature features using the curvature attention module to obtain target fused features; obtain a defect prediction result using the first defect detection head based on the target fused features; obtain a curvature prediction result using the second candidate curvature prediction head based on the RGB features; calculate a first loss value based on the defect prediction result and the real defect annotation; calculate a second loss value based on the curvature prediction result and the target curvature map; perform a weighted summation of the first loss value and the second loss value to obtain a target loss value; and optimize the network parameters of the initial teacher model based on the target loss value.

[0083] In an optional implementation, the second training module 503 is further configured to: calculate curvature confidence based on the curvature features using the curvature attention module; align the curvature features with the RGB features; obtain modulated curvature features based on the aligned curvature features and the curvature confidence; fuse the RGB features and the modulated curvature features based on an attention mechanism to obtain initial fused features; and perform residual connections on the initial fused features based on the RGB features to obtain the target fused features.

[0084] In an optional implementation, the second training module 503 is further configured to multiply the aligned curvature features element-wise with the curvature confidence to obtain the modulated curvature features.

[0085] In an optional implementation, the defect detection module 506 is further configured to input the RGB image of the metal surface of the metal ingot to be tested into the trained student model to obtain the defect detection result of the metal ingot surface RGB image.

[0086] The apparatus provided in this embodiment can perform the steps of the metal ingot surface defect detection method provided in Embodiment 1. To avoid repetition, it will not be described again.

[0087] The metal ingot surface defect detection device proposed in this embodiment improves the detection rate of small or low-contrast defects such as slag and burrs by fusing the RGB features in the RGB image of the metal surface with the three-dimensional geometric curvature features in the target curvature map. It adopts a geometric perception pre-training and a teacher-student knowledge distillation two-stage strategy to ensure detection accuracy while compressing the number of model parameters and improving inference speed.

[0088] Example 3 Furthermore, this disclosure provides a computer device including a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the metal ingot surface defect detection method described in Embodiment 1.

[0089] The device provided in this embodiment can perform the steps of the metal ingot surface defect detection method provided in Embodiment 1. To avoid repetition, it will not be described again.

[0090] Example 4 This disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the metal ingot surface defect detection method described in Embodiment 1.

[0091] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0092] The computer-readable storage medium provided in this embodiment can implement the metal ingot surface defect detection method provided in Embodiment 1. To avoid repetition, it will not be described again here.

[0093] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limitations; therefore, other examples of exemplary embodiments may have different values.

[0094] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0095] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects in metal ingots, characterized in that, The method includes: Acquire RGB images and 3D point clouds of the metal surface of a metal ingot, and obtain a target curvature map of the RGB image of the metal surface based on the 3D point cloud; The RGB image of the metal surface is used as input, and the target curvature map is used as a label to train the geometry perception network, resulting in a trained geometry perception network. The trained geometry perception network includes a first candidate RGB encoder and a first candidate curvature prediction head. The RGB image of the metal surface and the target curvature map are used as the main inputs, and the real defect annotations corresponding to the RGB image of the metal surface are used as the main labels. The RGB image of the metal surface is used as the secondary input, and the target curvature map is used as the secondary label to train the initial teacher model and obtain the target teacher model. The initial teacher model includes the first candidate RGB encoder, the curvature encoder, the curvature attention module, the first defect detection head, and the second candidate curvature prediction head. A preset RGB image of a metal surface is input into the target teacher model to obtain soft labels and their corresponding intermediate features. The target teacher model includes a trained first candidate RGB encoder, a trained curvature encoder, a trained curvature attention module, and a trained first defect detection head. The preset metal surface RGB image is used as input, the soft label and the real defect annotation corresponding to the preset metal surface RGB image are used as labels, and the intermediate feature corresponding to the soft label is used as supervision signal to train the student model and obtain the trained student model. The student model includes a second candidate RGB encoder and a second defect detection head. Surface defect detection of metal ingots is performed based on the trained student model.

2. The method for detecting surface defects of metal ingots according to claim 1, characterized in that, The step of obtaining the target curvature map of the metal surface RGB image based on the three-dimensional point cloud includes: Multi-scale normal vector estimation is performed on the three-dimensional point cloud to obtain the denoised normal vector; The Gaussian curvature and average curvature are calculated based on the denoised normal vector, and defect-sensitive curvature encoding is performed based on the Gaussian curvature and average curvature to obtain the defect-sensitive curvature. The defect-sensitive curvature is aligned with RGB edges based on the RGB image of the metal surface to obtain the target curvature map.

3. The method for detecting surface defects of metal ingots according to claim 2, characterized in that, The step of encoding defect-sensitive curvature based on the Gaussian curvature and the average curvature to obtain the defect-sensitive curvature includes: Obtain the absolute value of the Gaussian curvature, and transform the average curvature based on the activation function; The defect-sensitive curvature is obtained by multiplying the absolute value of the Gaussian curvature by the transformed average curvature.

4. The method for detecting surface defects of metal ingots according to claim 1, characterized in that, The process of using the RGB image of the metal surface and the target curvature map as the main input, the real defect annotations corresponding to the RGB image of the metal surface as the main label, and the RGB image of the metal surface as the secondary input and the target curvature map as the secondary label to train the initial teacher model and obtain the target teacher model includes: The first candidate RGB encoder obtains RGB features based on the RGB image of the metal surface, and the curvature encoder obtains curvature features based on the target curvature map. The RGB features and the curvature features are fused using the curvature attention module to obtain the target fused features; The first defect detection head obtains the defect prediction result based on the target fusion features, and the second candidate curvature prediction head obtains the curvature prediction result based on the RGB features; A first loss value is calculated based on the defect prediction result and the actual defect annotation, and a second loss value is calculated based on the curvature prediction result and the target curvature map. The target loss value is obtained by weighted summation of the first loss value and the second loss value, and the network parameters of the initial teacher model are optimized based on the target loss value.

5. The method for detecting surface defects of metal ingots according to claim 4, characterized in that, The process of fusing the RGB features and the curvature features through the curvature attention module to obtain the target fused features includes: The curvature confidence level is calculated based on the curvature features using the curvature attention module. Align the curvature feature with the RGB feature, and obtain the modulated curvature feature based on the aligned curvature feature and the curvature confidence. An initial fused feature is obtained by fusing the RGB features and the modulated curvature features based on an attention mechanism; The initial fusion feature is residually connected based on the RGB features to obtain the target fusion feature.

6. The method for detecting surface defects of metal ingots according to claim 5, characterized in that, The step of obtaining the modulated curvature features based on the aligned curvature features and the curvature confidence includes: The aligned curvature feature is multiplied element-wise with the curvature confidence to obtain the modulated curvature feature.

7. The method for detecting surface defects of metal ingots according to claim 1, characterized in that, The step of detecting surface defects in metal ingots based on the trained student model includes: The RGB image of the metal surface of the metal ingot to be tested is input into the trained student model to obtain the surface defect detection result of the metal ingot based on the RGB image of the metal surface to be tested.

8. A device for detecting surface defects in metal ingots, characterized in that, The device includes: The data processing module is used to acquire RGB images and three-dimensional point clouds of the metal surface of a metal ingot, and to obtain a target curvature map of the RGB image of the metal surface based on the three-dimensional point cloud. The first training module is used to train a geometry perception network by taking the RGB image of the metal surface as input and the target curvature map as a label, and to obtain a trained geometry perception network. The trained geometry perception network includes a first candidate RGB encoder and a first candidate curvature prediction head. The second training module is used to take the RGB image of the metal surface and the target curvature map as the main input, the real defect annotation corresponding to the RGB image of the metal surface as the main label, the RGB image of the metal surface as the secondary input, and the target curvature map as the secondary label, to train the initial teacher model and obtain the target teacher model. The initial teacher model includes the first candidate RGB encoder, the curvature encoder, the curvature attention module, the first defect detection head and the second candidate curvature prediction head. The third training module is used to input a preset RGB image of a metal surface into the target teacher model to obtain soft labels and their corresponding intermediate features. The target teacher model includes a trained first candidate RGB encoder, a trained curvature encoder, a trained curvature attention module, and a trained first defect detection head. The fourth training module is used to take the preset metal surface RGB image as input, the soft label and the real defect annotation corresponding to the preset metal surface RGB image as labels, and the intermediate features corresponding to the soft label as supervision signals to train the student model and obtain the trained student model. The student model includes a second candidate RGB encoder and a second defect detection head. The defect detection module is used to detect surface defects of metal ingots based on the trained student model.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, implements the method for detecting surface defects of metal ingots as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method for detecting surface defects of metal ingots as described in any one of claims 1 to 7.

Images