Precise detection method and system for surface defects of metal plate

Abstract

The invention relates to the technical field of metal surface defect detection, in particular to a metal plate surface defect precise detection method and system.The method comprises the steps that to-be-detected samples are extracted from metal plates of the same batch, the to-be-detected samples are sequentially moved to designated detection positions through a conveying device, and surface images are collected through a first industrial camera; pre-processing the image, and inputting the pre-processed image into the trained plate surface defect precision detection model to obtain a first defect detection result which comprises defect positions, defect types and defect severity. Accurately spraying fluorescent magnetic powder and synchronously applying a magnetic field according to the first defect detection result, shooting a magnetic powder coverage image by using a second industrial camera under an ultraviolet condition, and generating a second defect detection result through a magnetic powder defect detection model. And finally, generating a comprehensive result by comparing and correcting the preliminary detection result. According to the invention, computer vision detection and magnetic powder detection are combined, so that the precision and comprehensiveness of surface defect detection of the metal plate are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of metal surface defect detection, and particularly to a precise detection method and system for surface defects of metal plates. Background Art

[0002] Due to their excellent strength, durability, and workability, metal plates can meet diverse structural and performance requirements. Among them, ferromagnetic metal plates are widely used in fields such as construction, automotive, machinery manufacturing, shipbuilding, and power equipment due to their high mechanical strength, good magnetic conductivity, and excellent corrosion resistance. The types of ferromagnetic metal plates mainly include carbon steel, low alloy steel, and silicon steel. During the production process of ferromagnetic metal plates, cracks, pores, surface scratches, or other processing defects may occur before leaving the factory due to non-uniformity in the manufacturing process, raw material defects, stress concentration during heat treatment, etc. If these defects are not detected and repaired, they may affect the strength and durability of the product during subsequent use, and further lead to the failure of equipment or structures. Therefore, precise detection of surface and near-surface defects before the ferromagnetic metal plates leave the factory is a key step to ensure product quality.

[0003] Existing surface detection methods for ferromagnetic metal plates mainly include magnetic particle inspection, eddy current testing, and surface defect detection based on computer vision. Magnetic particle inspection is a commonly used surface defect detection method. Plates usually have a smooth planar shape, which makes it relatively simple to evenly coat magnetic powder on their surface. During magnetic particle inspection, magnetic powder needs to be evenly scattered on the surface to be inspected so that defects can be revealed by the aggregation of magnetic powder. For flat plates, magnetic powder can cover their surface more evenly without being difficult to coat or causing uneven magnetic powder accumulation as on complex-shaped or irregular surfaces, which may affect the detection effect. However, the operation of magnetic particle inspection is relatively cumbersome and complex, especially when inspecting large-area plates, it takes a long time and has a high cost. In addition, the accuracy of magnetic particle inspection is limited, and it is prone to missed detection of defects such as tiny surface cracks and pores.

[0004] Existing surface defect detection technologies based on computer vision usually rely on industrial cameras or sensors to collect surface images of metal plates and use image processing algorithms or deep learning models to identify and classify surface defects. They can quickly detect the surface of large-area plates and are particularly good at identifying obvious surface defects such as large scratches, corrosion, and discoloration. However, existing surface defect detection technologies based on computer vision have certain limitations, mainly reflected in the insufficient recognition accuracy of tiny defects (such as cracks and pores). In addition, vision detection methods mainly rely on surface images and are difficult to perform more in-depth multi-level analysis, thus limiting their accuracy and comprehensiveness in the detection of tiny and precise defects.

[0005] For this reason, a precise detection method and system for surface defects of metal sheets are proposed. Summary of the Invention

[0006] The purpose of the present invention is to provide a precise detection method and system for surface defects of metal sheets. Starting from the following main points, first, a batch of samples is extracted from ferromagnetic metal sheets of the same batch, and they are sequentially moved to the specified detection positions through a transmission device. The surface images are collected by a first industrial camera, and after preprocessing the images, they are input into a trained precise detection model for surface defects of metal sheets to generate a first defect detection result including the defect position, category, and severity. According to the first defect detection result, the amount of magnetic powder to be sprinkled is calculated, and fluorescent magnetic powder is sprayed and a synchronous magnetic field is applied to the defect area. Subsequently, a second industrial camera is used to take pictures of the defect positions under ultraviolet light conditions to obtain a magnetic powder coverage image, which is input into a trained magnetic powder defect detection and recognition model to generate a second defect detection result. Finally, the first detection result is corrected according to the second defect detection result to generate a comprehensive result. The present invention proposes a precise detection model for surface defects of metal sheets. Through multi-scale feature extraction and multi-task learning, combined with multi-channel input image data, it realizes the comprehensive detection of surface defects of metal sheets and improves the recognition accuracy of defects of different sizes. Through an automated magnetic powder spraying device, the spraying amount and angle of fluorescent magnetic powder can be accurately controlled according to the coordinates of the defect area, and at the same time, a magnetic field is applied to ensure the effective detection of the defect position, reduce external intervention, and improve the detection efficiency. Finally, a magnetic powder defect detection and recognition model is proposed. By combining multi-modal input of image and electromagnetic field data and using a weighted feature fusion layer and an attention mechanism for feature fusion, it effectively improves the accurate classification and confidence score of the defect position, category, and severity, and enhances the comprehensiveness and reliability of the detection.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A precise detection method for surface defects of metal sheets includes:

[0009] Extract a batch of metal sheets from the ferromagnetic metal sheets that have been produced, obtaining N metal sheets to be inspected; the metal sheets to be inspected are the ferromagnetic metal sheets produced in the same batch;

[0010] Move N of the metal sheets to be inspected to the designated inspection position in sequence through a transmission device; collect images of the surfaces of the metal sheets to be inspected at the designated inspection position by a first industrial camera to obtain M sets of initial images; preprocess the M sets of initial images to obtain M sets of preprocessed images; input the M sets of preprocessed images into a trained precision defect detection model for the surface of the sheet to obtain a first defect detection result; the first defect detection result includes a first defect position, a first defect category, and a first defect severity level.

[0011] Calculate the amount of magnetic powder to be sprayed on each defect area according to the first defect detection result, and evenly spray fluorescent magnetic powder on the defect area according to the first defect position corresponding to the defect area and the corresponding amount of magnetic powder to be sprayed, and apply a magnetic field to the defect area synchronously.

[0012] Use a second industrial camera to take pictures of the defect area under ultraviolet irradiation to obtain a magnetic powder coverage image; input the preprocessed magnetic powder coverage image into a trained magnetic powder defect detection and recognition model to obtain a second defect detection result; the second defect detection result includes a second defect position, a second defect category, and a second defect severity category; correct the first defect detection result according to the second defect detection result to generate a comprehensive result.

[0013] Further, moving N of the metal sheets to be inspected to the designated inspection position in sequence through the transmission device includes: conveying the metal sheets to be inspected from the loading area to the conveyor belt of the transmission device through an automatic loading and positioning device, so that each metal sheet to be inspected moves to the designated inspection position when the conveyor belt stops; the automatic loading and positioning device includes an automatic loading device and a positioning device, the automatic loading device is used to continuously convey the metal sheets to be inspected to the conveyor belt; the positioning device includes a laser projection device, an industrial camera, an analysis system, and a feedback system, the designated inspection position is located at a designated position on the conveyor belt, and the laser projection device is installed directly above the designated inspection position; project a virtual fixed frame with the same size as the metal sheet to be inspected onto the surface of the conveyor belt through the laser projection device; the first industrial camera is used to take pictures of the image at the designated inspection position when the conveyor belt stops conveying to obtain an image to be inspected; the analysis system is used to detect whether the edge of the metal sheet to be inspected coincides with the virtual fixed frame according to the image to be inspected to obtain an offset result; when the offset result is yes, calculate adjustment parameters; the feedback system is used to automatically adjust the conveyor belt according to the adjustment parameters when the offset result is yes, so that the edge of the metal sheet to be inspected coincides with the edge of the virtual fixed frame.

[0014] Further, the initial images are obtained by collecting images of the surface of the metal sheet to be inspected through the first industrial camera, and specifically, M groups of the initial images are obtained as follows: under different acquisition angles and illumination conditions, M groups of the initial images are acquired through the first industrial camera, and each group of the initial images corresponds to a combination of different angle parameters and illumination parameters.

[0015] Further, the training process of the precision detection model for the surface defects of the sheet includes the following steps:

[0016] Data preparation step: Collect surface images of metal sheets under different angles and illumination conditions to form M groups of input images; each group of the input images is labeled to obtain labeled input images, and the labeled input images contain label information of defect positions, defect categories, and defect severity levels.

[0017] Data preprocessing step: Perform data preprocessing on M groups of the labeled input images, including normalization and data augmentation, to obtain preprocessed labeled images.

[0018] Data division step: Divide the preprocessed labeled images into a first training set and a first validation set.

[0019] Training step: Input the first training set into the precision detection model for the surface defects of the sheet, and perform supervised learning using the defect positions, defect categories, and defect severity levels in the label information; during the training process, calculate the position error, classification error, and severity error based on the defect positions, defect categories, and defect severity levels in the label information; calculate the loss function based on the position error, classification error, and severity error, and update the model weight parameters through the backpropagation algorithm to obtain the first precision detection model for the surface defects of the sheet.

[0020] Validation step: Evaluate the performance of the first precision detection model for the surface defects of the sheet through the first validation set, calculate the loss function of the first validation set, and if the loss function value does not reach the preset convergence condition, adjust the hyperparameters of the first precision detection model for the surface defects of the sheet.

[0021] Repeat the training step and the validation step until the loss function of the first validation set reaches the preset convergence condition to obtain the precision detection model for the surface defects of the sheet.

[0022] Further, the precision detection model for the surface defects of the sheet includes: a multi-channel input layer, a feature extraction layer, a multi-scale feature extraction layer, a global feature extraction layer, a multi-task learning layer, and a confidence output layer.

[0023] The multi-channel input layer is used to receive M groups of input images under different angles and illumination conditions.

[0024] The feature extraction layer includes multiple convolutional layers and multiple batch normalization layers for generating feature maps; the convolutional layers are used to extract image features, and the batch normalization layers are used to normalize the output of each convolutional layer;

[0025] The multi-scale feature extraction layer uses convolutional kernels of different sizes to perform multi-scale processing on the feature maps to extract defect features at different scales and generate multi-scale feature maps;

[0026] The global feature extraction layer includes a global average pooling layer and a first fully connected layer; the global average pooling layer is used to summarize the global information of the multi-scale feature maps to obtain global features; and the first fully connected layer is used to convert the global features into global high-dimensional features;

[0027] The multi-task learning layer includes a defect location branch, a defect category branch, and a defect severity branch. The defect location branch is used to generate defect locations based on the global high-dimensional features; the defect category branch is used to generate defect categories based on the global high-dimensional features; the defect severity branch is used to generate defect severities based on the global high-dimensional features;

[0028] The confidence output layer includes a fully connected layer for comprehensively processing the defect location, defect category, and defect severity information to generate a comprehensive detection result and output the confidence score of each comprehensive detection result.

[0029] Next, a specific model structure in this embodiment is given. This model is one of the implementation solutions of the precision detection model for metal sheet surface defects. The precision detection model for metal sheet surface defects in this embodiment has the following structure: First, through a multi-channel input layer, the input dimension is M×H×W×C, where M is 15, representing the number of multi-angle image groups, H is the image height, W is the image width, and C is 3, representing the RGB channels. The model supports 15 groups of images to be input simultaneously. The feature extraction layer consists of three convolutional layers. The first convolutional layer uses 64 3x3 convolutional kernels, the second convolutional layer uses 128 3x3 convolutional kernels, and the third convolutional layer uses 256 3x3 convolutional kernels. All three convolutional layers use a stride of 1 and padding of 1, and batch normalization (Batch Normalization) and max pooling (2x2 pooling window) are applied after each convolutional layer. The multi-scale feature extraction layer uses convolutional kernels of different sizes, namely 128 convolutional kernels of 3x3, 5x5, and 7x7, in order to extract features at different scales. Then, the model performs global average pooling through the global feature extraction layer to reduce the number of parameters while retaining global information, and performs global feature processing through a fully connected layer with 512 neurons.

[0030] In the multi-task learning layer, the defect location branch consists of three fully connected layers. The first layer has 512 neurons, the second layer has 256 neurons, and the third layer has 128 neurons. The final output is 4 coordinate values, representing the position of the defect bounding box. The activation function is a linear activation. The defect category branch and the defect severity branch have a similar structure, each consisting of three fully connected layers. The first layer has 512 neurons, the second layer has 256 neurons, and the third layer has 128 neurons. The output of the category branch is 3 nodes, representing different defect categories. The activation function is Softmax. The severity branch also outputs 3 nodes, representing the severity of the defect (slight, medium, severe). The activation function is also Softmax. Finally, through the confidence output layer, combining the outputs of the above three branches, a confidence score for each comprehensive detection result is generated.

[0031] Further, inputting the M groups of the preprocessed images into the trained precision detection model for the surface defects of the sheet metal, the obtained first defect detection results include:

[0032] Inputting the M groups of the preprocessed images into the trained precision detection model for the surface defects of the sheet metal, M groups of defect detection results are respectively obtained. Each group of the defect detection results includes the defect location, defect category, defect severity, and detection confidence.

[0033] Processing the M groups of the defect detection results through confidence screening, position alignment, weighted voting for defect categories, and weighted averaging for severity, and integrating them into the first defect detection results. The first defect detection results include the final defect location, final defect category, and final defect severity of each to-be-inspected metal sheet.

[0034] Further, calculating the amount of magnetic powder to be sprayed on each defect area according to the first defect detection results, and uniformly spraying fluorescent magnetic powder on the defect area according to the corresponding first defect location and the corresponding amount of magnetic powder to be sprayed on the defect area. Synchronously applying a magnetic field to the defect area specifically includes:

[0035] Uniformly spraying fluorescent magnetic powder on the defect area through a magnetic powder spraying device, and synchronously applying a magnetic field. The magnetic powder spraying device includes:

[0036] A spray head, used to automatically adjust the opening degree and spraying angle of the nozzle according to the defect area, so that the fluorescent magnetic powder uniformly covers the defect area.

[0037] A robotic arm, used to support and move the spray head, so that the spray head moves to the defect area.

[0038] Magnetic powder storage and transportation system, including a magnetic powder storage tank and a transportation pipeline. A weighing sensor is provided in the magnetic powder storage tank, and the weighing sensor is used to monitor the remaining amount of magnetic powder in real time. The transportation pipeline is used to send fluorescent magnetic powder from the magnetic powder storage tank to the spray head;

[0039] An electromagnet module for synchronously applying a magnetic field while spraying fluorescent magnetic powder;

[0040] The magnetic powder spraying device further includes: a calculation and analysis module for calculating the amount of magnetic powder to be sprayed in each defect area according to the first defect detection result; obtaining the spraying density, spraying times, and spraying time according to the first defect detection result, and controlling the operating parameters of the spray head, the robotic arm, and the electromagnet module according to the spraying density, the spraying times, the spraying time, and the amount of magnetic powder to be sprayed.

[0041] Further, the training process of the magnetic powder defect detection and recognition model includes the following steps:

[0042] Collect surface images of metal plates covered with fluorescent magnetic powder from different angles to form multiple groups of magnetic powder input images, and simultaneously collect electromagnetic field response data during and after the spraying of fluorescent magnetic powder; annotate the multiple groups of magnetic powder input images, and the annotation content includes the defect position, defect category, and defect severity to obtain annotated magnetic powder images. Combine the annotated magnetic powder images with the corresponding electromagnetic field response data respectively to obtain multimodal data; perform data preprocessing on the multimodal data to obtain preprocessed multimodal data; divide the preprocessed multimodal data into a second training set and a second validation set; obtain the trained magnetic powder defect detection and recognition model according to the second training set and the second validation set.

[0043] Further, the magnetic powder defect detection and recognition model includes:

[0044] An input layer for receiving multimodal input data; the multimodal input data includes image input and electromagnetic field data input;

[0045] A feature extraction layer, including an image feature extraction layer and an electromagnetic field feature extraction layer; the image feature extraction layer includes multiple convolutional layers for extracting the features of the image input and generating image features; the electromagnetic field feature extraction layer is used to extract the features of the electromagnetic field data input and generate electromagnetic field features;

[0046] The feature fusion layer, including a weighted feature fusion layer and an attention mechanism layer, is used to perform fusion processing on the image features and the electromagnetic field features; the weighted feature fusion layer adjusts the weight ratio of the image features and the electromagnetic field features by learning the weights of different modality input data in the multi-modal input data, and generates fusion features; the attention mechanism layer assigns weights to different features of the fusion features according to the feature importance;

[0047] The global feature extraction layer, including a global average pooling layer, is used to transform the image features and the electromagnetic field features into global features;

[0048] The multi-task learning layer, including a defect location branch, a defect category branch, and a defect severity branch; the defect location branch is used to generate a defect location based on the global features; the defect category branch is used to generate a defect category based on the global features; the defect severity branch is used to generate a severity based on the global features;

[0049] The confidence output layer receives the defect location, the defect category, and the defect severity from the multi-task learning layer, and generates a comprehensive detection result; and generates a confidence score for each comprehensive detection result through a fully connected layer.

[0050] A precision detection system for surface defects of metal sheets, comprising:

[0051] The to-be-inspected sheet transmission module is used to extract a batch of metal sheets from the ferromagnetic metal sheets that have been produced, and obtain N to-be-inspected metal sheets; the to-be-inspected metal sheets are the ferromagnetic metal sheets produced in the same batch; and move the N to-be-inspected metal sheets to the designated detection position in sequence through a transmission device;

[0052] The image acquisition module is used to perform image acquisition on the surface of the to-be-inspected metal sheet at the designated detection position through a first industrial camera, and obtain M groups of initial images;

[0053] The preprocessing module for the surface image of the sheet is used to preprocess the M groups of initial images to obtain M groups of preprocessed images;

[0054] The surface defect detection and recognition module is used to input the M groups of preprocessed images into the trained precision detection model for surface defects of the sheet to obtain a first defect detection result; the first defect detection result includes a first defect location, a first defect category, and a first defect severity;

[0055] A magnetic powder spraying module, configured to calculate the amount of magnetic powder to be sprayed on each defect area according to the first defect detection result, uniformly spray fluorescent magnetic powder on the defect area according to the corresponding first defect position and the corresponding amount of magnetic powder to be sprayed in the defect area, and simultaneously apply a magnetic field to the defect area;

[0056] A fluorescence imaging module, configured to use a second industrial camera to photograph the defect area under ultraviolet irradiation to obtain a magnetic powder coverage image;

[0057] A magnetic powder image preprocessing module, configured to preprocess the magnetic powder coverage image;

[0058] A magnetic powder detection and recognition module, configured to input the preprocessed magnetic powder coverage image into a trained magnetic powder defect detection and recognition model to obtain a second defect detection result; the second defect detection result includes a second defect position, a second defect category, and a second defect severity category;

[0059] A result correction module, configured to correct the first defect detection result according to the second defect detection result to generate a comprehensive result.

[0060] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0061] 1. The present invention proposes a preliminary defect detection method based on a precise detection model for surface defects of metal sheets. Combining multi-scale feature extraction and multi-task learning, it can detect surface defects of metal sheets more comprehensively and accurately. By collecting images under different angles and lighting conditions through a multi-channel input layer, the adaptability of the model to complex surface environments is improved; the feature extraction layer extracts detailed information to ensure that the model can identify tiny defects; the multi-scale feature extraction layer captures defect information of different sizes and shapes through convolutional kernels of different sizes, especially having a good recognition effect on small defects. The global feature extraction layer further integrates global information, improving the model's understanding of the overall defect pattern and ensuring accuracy. The multi-task learning layer simultaneously predicts the position, type, and severity of defects, making the subsequent further magnetic powder detection for defect areas more efficient and targeted. Secondly, the confidence score provides a reliability index for each detection result. Generally, this method effectively improves the accuracy and comprehensiveness of surface defect detection of metal sheets.

[0062] 2. The present invention proposes an automated magnetic powder spraying method by integrating a calculation and analysis module, a spray head, a robotic arm, a magnetic powder storage and conveying system, and an electromagnet module. It can accurately control the magnetic powder spraying process according to the size and position of the defect area, ensure uniform coverage of the magnetic powder in the defect area, and simultaneously apply a magnetic field, thereby improving the accuracy and consistency of detection. By automatically adjusting the opening degree and spraying angle of the spray head, it avoids magnetic powder waste and uneven spraying, ensuring sufficient detection of each defect position. The combination of the robotic arm and the calculation and analysis module makes the spraying process intelligent and automated, flexibly adjusting the spraying parameters according to the specific situation of the defect area, reducing operation intervention, and improving work efficiency and detection accuracy. In addition, the weighing sensor monitors the remaining amount of magnetic powder in real time to ensure the continuous and stable operation of the system during the detection process, further guaranteeing the accuracy of subsequent detection results. This automated magnetic powder spraying method based on the magnetic powder spraying device solves the problems of long time consumption and cumbersome operation in traditional magnetic powder detection.

[0063] 3. The present invention proposes a magnetic powder defect detection and recognition model. Combining multi-modal inputs of image data and electromagnetic field data, the model can detect surface defects of metal sheets more comprehensively and accurately. By the feature extraction layer, image features and electromagnetic field features are respectively extracted, which can capture deep-level information that cannot be obtained by traditional single-modal detection methods. The weighted feature fusion layer and the attention mechanism layer further optimize the fusion of different modal features, ensuring that the model can dynamically allocate weights according to the importance of features, improving the detection accuracy of key defects. The global feature extraction layer ensures the comprehensive integration of features, enabling the model to have stronger detection and recognition capabilities globally. The multi-task learning layer simultaneously predicts the position, category, and severity of defects, greatly improving the detection efficiency and accuracy. Finally, the confidence output layer generates a confidence score for each detection result, enhancing the reliability of the detection results and reducing the risks of false alarms and missed detections. This model combines multi-modal data, feature fusion, and multi-task learning, significantly improving the accuracy and comprehensiveness of the detection of surface defects of metal sheets. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flowchart of a method for precise detection of surface defects of metal sheets provided by an embodiment of the present invention;

[0065] Figure 2 It is a schematic diagram of the automatic transmission and positioning of a metal sheet to be inspected provided by an embodiment of the present invention;

[0066] Figure 3 It is a structural diagram of a precise detection model for surface defects of a sheet provided by an embodiment of the present invention;

[0067] Figure 4 It is a structural diagram of a magnetic powder defect detection and recognition model provided by an embodiment of the present invention;

[0068] Figure 5 This is a structural diagram of a precision detection system for surface defects of metal sheets provided by an embodiment of the present invention.

[0069] In the figure: 1. Metal sheet to be inspected; 2. Conveyor belt; 3. Laser projection device; 4. First industrial camera; 5. Virtual fixed frame; 6. Robot arm. Specific implementation manners

[0070] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0071] Magnetic particle testing is a technique for defect detection that utilizes the magnetic property changes of ferromagnetic materials under the action of an external magnetic field. When a ferromagnetic material is in a magnetic field, if there are cracks or defects on the material surface, the distribution of magnetic flux will be abnormal, resulting in local changes in magnetic induction intensity. By applying magnetic-sensitive particles (such as ferromagnetic powders) on the material surface, these changes can be revealed. The inspector can use visible light or fluorescence means to observe the aggregation of magnetic particles at the defects, thereby judging the surface defects of the material and further evaluating its quality.

[0072] Embodiment 1

[0073] Factory A applied a precision detection method for surface defects of metal sheets to detect the surface defects of ferromagnetic metal sheets and ensure that the quality meets the standards when leaving the factory.

[0074] Refer to Figure 1 In S10 of, a batch of metal sheets is randomly selected from the ferromagnetic metal sheets completed in production to obtain N metal sheets to be inspected; the metal sheets to be inspected are the ferromagnetic metal sheets produced in the same batch; in this embodiment, the metal sheets to be inspected are uncoated carbon steel sheets produced in the same batch, that is, the production standard indicators and production processes of the metal sheets to be inspected are completely the same, including the same raw material components, heat treatment processes, dimensional specifications, and surface treatment requirements. The N metal sheets to be inspected are all rectangular sheets of the same size. In this embodiment, the number of samples selected N is 100.

[0075] Refer to Figure 1 In S20 of, the N metal sheets to be inspected are sequentially moved to the designated detection position through a transmission device and fixed to ensure that each metal sheet is in an accurate position during detection, and the designated detection position is the position for subsequent image acquisition and spraying magnetic particles for magnetic particle testing.

[0076] Further, moving the N metal sheets to be inspected to the specified inspection position in sequence through the transfer device includes: Refer to Figure 2 , conveying the metal sheet 1 to be inspected from the loading area to the conveyor belt 2 of the transfer device through an automatic loading and positioning device, so that the metal sheet 1 to be inspected moves to the specified inspection position when the conveyor belt 2 stops; the automatic loading and positioning device includes an automatic loading device and a positioning device, and the automatic loading device is used to continuously convey the metal sheet 1 to be inspected to the conveyor belt 2; the positioning device includes a laser projection device 3, the first industrial camera 4, an analysis system and a feedback system, the specified inspection position is located at a specified position on the conveyor belt 2, and the laser projection device 3 is installed directly above the specified inspection position; a virtual fixed frame 5 having the same size as the metal sheet 1 to be inspected is projected onto the surface of the conveyor belt 2 through the laser projection device 3; Refer to Figure 2 , the rectangular frame enclosed by the dotted line area on the conveyor belt 2 is the virtual fixed frame 5; the first industrial camera 4 is used to capture an image at the specified inspection position when the conveyor belt 2 stops conveying to obtain an image to be inspected; the analysis system is used to detect whether the edge of the metal sheet 1 to be inspected coincides with the virtual fixed frame 5 according to the image to be inspected to obtain an offset result; when the offset result is yes, calculate adjustment parameters; the feedback system is used to automatically adjust the conveyor belt 2 according to the adjustment parameters when the offset result is yes, so that the edge of the metal sheet 1 to be inspected coincides with the edge of the virtual fixed frame 5. Among them, the first industrial camera 4 is installed on a movable robotic arm 6; repeat the above loading and positioning operations N times.

[0077] The precise positioning of the metal sheet 1 to be inspected on the conveyor belt 2 is realized through the automatic loading and positioning device, ensuring that each metal sheet can accurately move to the specified inspection position when the conveyor belt 2 stops. With the help of the laser projection device 3, a virtual fixed frame 5 having the same size as the metal sheet is generated, and the edge position of the metal sheet 1 to be inspected is detected in real time through the first industrial camera 4 and the analysis system to ensure that the sheet is completely aligned with the virtual fixed frame 5. If there is an offset, the feedback system will automatically correct the position of the conveyor belt 2 according to the calculated adjustment parameters to realign the sheet. This high-precision positioning mechanism provides a stable and reliable basis for subsequent automated image acquisition and magnetic particle inspection, avoiding detection errors caused by sheet position offset, thereby greatly improving the accuracy and efficiency of inspection.

[0078] Refer to Figure 1 In S30 of , image acquisition is performed on the surface of the metal sheet 1 to be inspected at the specified inspection position through the first industrial camera 4 to obtain M groups of initial images; among them, the first industrial camera 4 is installed on a movable robotic arm 6

[0079] Further, image acquisition is performed on the surface of the to-be-inspected metal sheet 1 by the first industrial camera 4 to obtain M groups of the initial images. Specifically: under different acquisition angles and illumination conditions, M groups of the initial images are obtained by the first industrial camera 4, and each group of the initial images corresponds to a different combination of angle parameters and illumination parameters.

[0080] By performing image acquisition on the to-be-inspected metal sheet 1 under different acquisition angles and illumination conditions, the fine defects on the surface of the sheet can be comprehensively covered and captured. Each group of initial images corresponds to a different combination of angle parameters and illumination parameters. This multi-angle and multi-illumination image acquisition method can effectively reduce the detection blind spots that may exist under a single angle or illumination condition, ensure that all potential surface defects can be accurately identified and detected, thereby improving the comprehensiveness and reliability of the detection, and providing richer and more accurate image data for subsequent defect identification.

[0081] In this embodiment, M is 15. Referring to Table 1, the M groups of initial images are specifically the following 15 combinations of different shooting angles and illumination conditions.

[0082] Table 1. Initial Image Acquisition Parameter Table

[0083] Group Sequence Shooting Angle Illumination Intensity (Lux) 1 Vertical (90 degrees) 1000 2 Shooting at 60-degree Oblique Angle 1000 3 Shooting at 45-degree Oblique Angle 1000 4 Shooting at 30-degree Oblique Angle 1000 5 Shooting at 15-degree Oblique Angle 1000 6 Vertical (90 degrees) 500 7 Shooting at 60-degree Oblique Angle 500 8 Shooting at 45-degree Oblique Angle 500 9 Shooting at 30-degree Oblique Angle 500 10 Shooting at 15-degree Oblique Angle 500 11 Vertical (90 degrees) 200 12 Shooting at 60-degree Oblique Angle 200 13 Shooting at 45-degree Oblique Angle 200 14 Shooting at 30-degree Oblique Angle 200 15 Shooting at 15-degree Oblique Angle 200

[0084] Further, the training process of the fine detection model for the surface defects of the sheet includes the following steps:

[0085] Data preparation step: Collect surface images of metal sheets from different angles and illumination conditions to form M groups of input images; each group of the input images is labeled to obtain labeled input images, and the labeled input images contain label information on the defect position, defect category, and defect severity.

[0086] Data preprocessing step: Perform data preprocessing on the M groups of labeled input images, including normalization and data augmentation, to obtain preprocessed labeled images.

[0087] Data division step: Divide the preprocessed labeled images into a first training set and a first validation set.

[0088] Training step: Input the first training set into the fine detection model for the surface defects of the sheet, and perform supervised learning using the defect position, defect category, and defect severity in the label information; during the training process, calculate the position error, classification error, and severity error according to the defect position, defect category, and defect severity in the label information; calculate the loss function based on the position error, classification error, and severity error, and update the model weight parameters through the backpropagation algorithm to obtain the first fine detection model for the surface defects of the sheet.

[0089] Verification step: Evaluate the performance of the first sheet surface defect precision detection model through the first verification set, calculate the loss function of the first verification set, and if the loss function value does not reach the preset convergence condition, adjust the hyperparameters of the first sheet surface defect precision detection model;

[0090] Repeat the training step and the verification step until the loss function of the first verification set reaches the preset convergence condition to obtain the sheet surface defect precision detection model.

[0091] The training process ensures the high precision and robustness of the sheet surface defect detection model through systematic data collection, annotation, and model optimization steps. By collecting images from different angles and lighting conditions, the model can comprehensively learn the defect manifestations in various complex scenarios, thereby improving the recognition ability for different types and forms of defects. The accuracy of image annotation and the label information covering the defect location, category, and severity enable the model to refine its understanding of defects during the training process. Data preprocessing helps improve the stability of the model in dealing with data noise and environmental changes. Through supervised learning, the model can finely adjust the error for each type of defect and gradually optimize its weight parameters during backpropagation. Verification and hyperparameter adjustment further ensure the performance convergence of the model, ultimately achieving high detection accuracy and low false detection rate of the model in practical applications. This process significantly improves the adaptability and detection efficiency of the model in complex production environments.

[0092] Furthermore, as Figure 3 shown, the sheet surface defect precision detection model includes: a multi-channel input layer, a feature extraction layer, a multi-scale feature extraction layer, a global feature extraction layer, a multi-task learning layer, and a confidence output layer;

[0093] The multi-channel input layer is used to receive M groups of input images from different angles and lighting conditions;

[0094] The feature extraction layer includes a plurality of convolutional layers and a plurality of batch normalization layers for generating feature maps; the convolutional layers are used to extract image features, and the batch normalization layers are used to normalize the output of each convolutional layer;

[0095] The multi-scale feature extraction layer performs multi-scale processing on the feature maps using convolutional kernels of different sizes to extract defect features at different scales and generate multi-scale feature maps;

[0096] The global feature extraction layer includes a global average pooling layer and a first fully-connected layer; the global average pooling layer is used to summarize the global information of the multi-scale feature map to obtain global features; and the first fully-connected layer is used to convert the global features into global high-dimensional features;

[0097] The multi-task learning layer includes a defect location branch, a defect category branch, and a defect severity branch. The defect location branch is used to generate the defect location based on the global high-dimensional features; the defect category branch is used to generate the defect category based on the global high-dimensional features; the defect severity branch is used to generate the defect severity based on the global high-dimensional features;

[0098] The confidence output layer includes a fully-connected layer, which is used to comprehensively process the defect location, the defect category, and the defect severity information to generate a comprehensive detection result and output the confidence score of each comprehensive detection result.

[0099] The precision detection model for sheet surface defects receives M groups of images from different angles and lighting conditions through the multi-channel input layer, ensuring that the model can capture multi-dimensional information on the sheet surface, thereby improving the comprehensiveness and accuracy of defect recognition. The feature extraction layer efficiently extracts and standardizes image features through multiple convolutional layers and batch normalization layers, ensuring that the model maintains a stable feature extraction ability under different batches and conditions. The multi-scale feature extraction layer further extracts defect features at each scale through convolutional kernels of different sizes, ensuring that defects of different sizes and shapes can be captured. The global feature extraction layer integrates and performs high-dimensional processing on the multi-scale features through global average pooling and fully-connected layers to form a global feature representation, providing strong support for subsequent multi-task learning. In the multi-task learning layer, the model can simultaneously predict the specific location, category, and severity of defects, ensuring the accuracy and comprehensiveness of detection. Finally, the confidence output layer comprehensively processes various types of information about the defects, generates a confidence score, and provides a reliability assessment of the detection results. This design of multi-task learning and multi-scale feature fusion enables the model to achieve efficient, accurate, and comprehensive surface defect detection in practical applications, significantly improving the reliability and robustness of detection.

[0100] Refer to Figure 1 In S40 of, preprocess the M groups of the initial images respectively, including denoising, contrast enhancement, and geometric correction, to obtain M groups of preprocessed images; perform the first-stage defect recognition and classification: input the M groups of the preprocessed images into the trained precision detection model for sheet surface defects for preliminary defect recognition. The model automatically detects the defect areas in the images based on deep learning to obtain the first defect detection result, which specifically includes:

[0101] First defect location: Identify the specific coordinates where the defect is located;

[0102] The first defect category: classified as defects including different types such as cracks, pores, and scratches;

[0103] The first defect severity: evaluate its severity according to characteristics such as the size and shape of the defect; including minor defects, medium defects, and severe defects.

[0104] Detection confidence: the credibility score of the recognition results of the defect position, defect category, and severity in the defect detection results of each defect by the precision detection model for surface defects of the sheet metal, expressed as a score from 0 to 1, close to 1 indicating high confidence, and close to 0 indicating low confidence.

[0105] Furthermore, inputting the M groups of the preprocessed images into the trained precision detection model for surface defects of the sheet metal, the obtained first defect detection results include: inputting the M groups of the preprocessed images into the trained precision detection model for surface defects of the sheet metal, respectively obtaining M groups of defect detection results, and each group of the defect detection results includes defect position, defect category, defect severity, and detection confidence; integrating the M groups of the defect detection results through confidence screening, position alignment, defect category weighted voting, and severity weighted average processing to obtain the first defect detection results; the first defect detection results include the final defect position, final defect category, and final defect severity of each of the to-be-tested metal sheets 1.

[0106] By inputting M groups of preprocessed images into the trained precision detection model for surface defects of the sheet metal, multiple groups of defect detection results can be obtained from different angles and lighting conditions. These detection results contain the specific position, category, severity, and detection confidence of the defects. Through processing steps such as confidence screening, position alignment, category weighted voting, and severity weighted average, it is ensured that the model synthesizes information from multiple angles and conditions and outputs more accurate defect detection results. This method can effectively avoid misdetection or missed detection problems under a single angle or lighting condition, improving the robustness and reliability of defect detection. The integrated first defect detection results can more accurately reflect the defect position, category, and its severity of each metal sheet, thus providing a reliable basis for further detection and processing, significantly enhancing the comprehensiveness and accuracy of detection.

[0107] As a feasible implementation method in this embodiment, input 15 groups of the preprocessed images (each group includes 100 surface images of the to-be-tested metal sheets 1) into the trained precision detection model for surface defects of the sheet metal, and respectively obtain 15 groups of defect detection results through model prediction. Each group contains 100 defect detection results of the to-be-tested metal sheets 1 (if there is no defect, it is a null value). The defect detection results include defect position, defect category, defect severity, and detection confidence;

[0108] In this embodiment, to ensure that there is no overlap among the 15 groups of preprocessed images (each group includes 100 surface images of the metal sheets 1 to be inspected), the following measures are taken: First, a unique identification number is assigned to each metal sheet 1 to be inspected to ensure that the image acquisition process for each sheet is unique. Second, through the automatic loading and positioning device, the movement and positioning of the sheets on the conveyor belt 2 are precisely controlled to ensure that each acquisition is carried out only at the specified time and position, avoiding overlap with the image acquisitions of other sheets. Meanwhile, during the image acquisition process, a time interval control and a coordinate-based region division method are adopted to ensure that the regions and angles of each group of image acquisitions do not overlap with each other, so as to ensure that the surface of the sheet in each group of images is independent and the same surface region will not be repeatedly acquired.

[0109] The M groups of the defect detection results are processed and integrated through the following steps to generate the final first defect detection result:

[0110] Confidence screening: According to the set confidence threshold (0.8 in this embodiment), the defect detection results with a confidence exceeding the preset threshold are screened out, and the defect detection results with a confidence not exceeding the preset threshold are excluded;

[0111] Position alignment: According to the position of each defect, the multi-angle detection results are aligned, and the same defect regions detected in different groups are integrated into one defect;

[0112] Defect category weighted voting: The defect categories in the 15 groups of detection results at the same defect position are weighted and voted. Each category is weighted according to its corresponding confidence, and the category with a higher confidence gets a higher score. The final defect category of this region is determined according to the voting result;

[0113] Severity weighted average processing: The severities of the defects detected multiple times are weighted and averaged to finally determine the severity of this region; for the severity of each defect, the severity of each group in the 15 groups of detection results is quantified (for example, minor defect = 1, medium defect = 2, severe defect = 3), and weighted average is carried out according to the confidence of each group of detections.

[0114] According to the weighted average value, it is judged whether the final severity belongs to a minor, medium or severe defect. For example, if the weighted average value is between 1.0 and 1.5, it is a minor defect, between 1.5 and 2.5 is a medium defect, and greater than 2.5 is a severe defect.

[0115] After the integration is completed, the first defect detection result is obtained, which contains the final defect detection results of 100 metal sheets 1 to be inspected, and each result includes:

[0116] Final defect position: The defect coordinates after position alignment;

[0117] Final defect category: The defect category determined by weighted voting;

[0118] Final defect severity: The defect severity determined by weighted average processing;

[0119] Confidence level: The confidence score for each final defect result.

[0120] See Figure 1 In S50, calculate the amount of magnetic powder to be sprinkled on each defect area according to the first defect detection result. For example, if there are 208 defect areas, there will be 208 corresponding amounts of magnetic powder to be sprinkled. Sprinkle fluorescent magnetic powder evenly on the defect areas according to the first defect position corresponding to the defect area and the corresponding amount of magnetic powder to be sprinkled, and apply a magnetic field to the defect areas synchronously; ensure the aggregation of fluorescent magnetic powder in the defect areas to facilitate subsequent detection.

[0121] See Figure 1 In S60, use a second industrial camera to take pictures of the defect areas under ultraviolet irradiation to obtain a magnetic powder coverage image;

[0122] See Figure 1 In S70, preprocess the magnetic powder coverage image to obtain a preprocessed magnetic powder coverage image; perform defect recognition and classification in the second stage: input the preprocessed magnetic powder coverage image into a trained magnetic powder defect detection and recognition model, which can further accurately identify and classify defects to obtain a second defect detection result; obtain the second defect detection result, specifically including:

[0123] Second defect position: Further confirm the specific coordinates of the defect.

[0124] Second defect category: Make a more refined classification of the defect type.

[0125] Second defect severity category: Based on the fluorescence display effect of the defect, accurately evaluate the depth and severity of the defect.

[0126] Example of output of the second defect detection result:

[0127] {id:"1"

[0128] Label1:"Crack"

[0129] Label2:"Severe"

[0130] location:"[96.4444643.190468116.3057171.49247]"

[0131] score:"0.99343586"}

[0132] Further, calculate the amount of fluorescent magnetic powder to be sprayed on each of the defect regions according to the first defect detection result, and uniformly spray the fluorescent magnetic powder on the defect regions according to the corresponding first defect positions and the corresponding amounts of fluorescent magnetic powder to be sprayed on the defect regions. Synchronously applying a magnetic field to the defect regions specifically includes:

[0133] Uniformly spray the fluorescent magnetic powder on the defect regions through a magnetic powder spraying device, and synchronously apply a magnetic field; the magnetic powder spraying device includes:

[0134] A spray head for automatically adjusting the opening degree and spraying angle of the nozzle according to the defect region, so that the fluorescent magnetic powder uniformly covers the defect region;

[0135] A robotic arm for supporting and moving the spray head to move the spray head to the defect region;

[0136] A magnetic powder storage and conveying system, including a magnetic powder storage tank and a conveying pipeline. The magnetic powder storage tank is equipped with a weighing sensor for real-time monitoring of the remaining amount of magnetic powder, and the conveying pipeline is used to send the fluorescent magnetic powder from the magnetic powder storage tank to the spray head;

[0137] An electromagnet module for synchronously applying a magnetic field while spraying the fluorescent magnetic powder;

[0138] The magnetic powder spraying device further includes: a calculation and analysis module for calculating the amount of fluorescent magnetic powder to be sprayed on each of the defect regions according to the first defect detection result; obtaining the spraying density, spraying times, and spraying time according to the first defect detection result, and controlling the operating parameters of the spray head, the robotic arm, and the electromagnet module according to the spraying density, the spraying times, the spraying time, and the amount of fluorescent magnetic powder to be sprayed.

[0139] Table 2. Calculation Table of Magnetic Powder Spraying Parameters for Defect Regions

[0140]

[0141] Referring to Table 2, the defect area is calculated based on the coordinate information of the first defect position in the first defect detection result, and is used to determine the actual surface area of the defect. The spraying density represents the amount of magnetic powder required per unit area, and the specific value depends on the nature of the defect or surface characteristics. For defects of different severity levels (slight, medium, and severe), the number of spraying times will be adjusted accordingly. More severe defects usually require more magnetic powder and multiple sprayings to ensure that the fluorescent magnetic powder can fully cover the defect area and enhance the detection effect. The final amount of magnetic powder to be sprayed (g) is calculated by the following formula: Amount of magnetic powder to be sprayed = Defect area × Spraying density × Number of spraying times; The formula for the spraying time is: Spraying time = Amount of magnetic powder to be sprayed / Spraying rate; where the spraying rate is a preset value based on the defect detection result of each defect area.

[0142] By precisely controlling the magnetic powder spraying and synchronously applying a magnetic field, efficient and accurate defect detection can be achieved. First, the amount of magnetic powder to be sprayed in each defect area is calculated based on the first defect detection result, ensuring that the fluorescent magnetic powder is only sprayed on the defect area, avoiding unnecessary waste and coverage. The spray head in the magnetic powder spraying device can automatically adjust the opening degree and spraying angle of the nozzle according to the size and position of the defect area, so that the fluorescent magnetic powder evenly covers the defect area. The robotic arm can flexibly move the spray head to ensure that the magnetic powder is accurately sprayed on the specific position of each defect. At the same time, the electromagnet module synchronously applies a magnetic field during the spraying process, further improving the detection sensitivity of the defect area. By accurately calculating the amount of magnetic powder to be sprayed, the number of spraying times, and the duration through the calculation and analysis module, the whole process is highly automated, reducing errors in the operation process and improving the detection efficiency. In addition, the weighing sensor in the magnetic powder storage and conveying system monitors the amount of magnetic powder in real time to ensure the continuity and stability of the spraying process. The overall design improves the accuracy, efficiency, and reliability of the detection system, effectively ensuring the accuracy of defect marking and providing a strong foundation for subsequent detection.

[0143] Furthermore, the training process of the magnetic powder defect detection and recognition model includes the following steps:

[0144] Collect surface images of the metal sheet covered with fluorescent magnetic powder from different angles to form multiple groups of magnetic powder input images. The image acquisition parameters (shooting angles) of the multiple groups of magnetic powder input images are shown in Table 1. At the same time, collect the electromagnetic field response data during and after the spraying process of the fluorescent magnetic powder; label the multiple groups of the magnetic powder input images, and the labeling content includes the defect position, defect category, and defect severity to obtain labeled magnetic powder images. Combine the labeled magnetic powder images with the corresponding electromagnetic field response data respectively to obtain multimodal data; perform data preprocessing on the multimodal data to obtain preprocessed multimodal data; divide the preprocessed multimodal data into a second training set and a second validation set; obtain the trained magnetic powder defect detection and recognition model according to the second training set and the second validation set.

[0145] The magnetic powder defect detection and recognition model undergoes a systematic training process. Using the surface images of the metal sheet covered with fluorescent magnetic powder from different angles and the electromagnetic field response data, it forms multimodal data for joint training. The combination of multimodal data (image data and electromagnetic field data) enables the model to comprehensively analyze the characteristics of the defect area, relying not only on the visual information on the surface but also capturing the changes in the electromagnetic field response, thereby improving the detection accuracy for different types of defects. The data preprocessing step further optimizes the quality of the input data, enabling the model to maintain high robustness in complex environments. Through the training and validation of the multimodal data, the model can effectively identify the position, category, and severity of defects, improving the accuracy and reliability of detection. The use of this multimodal data greatly enhances the model's comprehensive recognition ability for metal surface defects, providing a more accurate decision-making basis for subsequent detection and quality control.

[0146] Furthermore, as Figure 4 shown, the magnetic powder defect detection and recognition model includes:

[0147] An input layer for receiving multimodal input data; the multimodal input data includes image input and electromagnetic field data input;

[0148] A feature extraction layer, including an image feature extraction layer and an electromagnetic field feature extraction layer; the image feature extraction layer includes multiple convolutional layers for extracting the features of the image input and generating image features; the electromagnetic field feature extraction layer is used to extract the features of the electromagnetic field data input and generate electromagnetic field features;

[0149] The feature fusion layer, including a weighted feature fusion layer and an attention mechanism layer, is used to perform fusion processing on the image features and the electromagnetic field features; the weighted feature fusion layer adjusts the weight ratio of the image features and the electromagnetic field features by learning the weights of different modal input data in the multi-modal input data, and generates fused features; the attention mechanism layer assigns weights to different features of the fused features according to the feature importance;

[0150] The global feature extraction layer, including a global average pooling layer, is used to convert the image features and the electromagnetic field features into global features;

[0151] The multi-task learning layer includes a defect location branch, a defect category branch, and a defect severity branch; the defect location branch is used to predict the specific location of the defect based on the global features and generate a defect location; the defect category branch is used to classify the specific type of the defect based on the global features and generate a defect category; the defect severity branch is used to predict the severity level of the defect based on the global features and generate a severity; specifically, the defect location branch further extracts spatial features through three convolutional layers to enhance the precise positioning of the defect location, and outputs the specific location parameters of the defect through a regression layer, including the center coordinates (x, y) and the width and height (w, h), and finally generates the actual defect location coordinates through a linear activation function. The defect category branch consists of two convolutional layers and a fully connected layer, which is responsible for classifying the defect and outputs the probability distribution of each category through a Softmax activation function, and finally obtains the defect category. The defect severity branch consists of two convolutional layers and a fully connected layer, which is used to predict the severity level of the defect, and the output result is processed through a Softmax activation function. The output results of the three branches are merged into a vector through a splicing layer, including information on location, category, and severity, and then are passed into the confidence output layer.

[0152] The confidence output layer receives the defect location, the defect category, and the defect severity from the multi-task learning layer, and generates a confidence score for each detection result through a fully connected layer. Specifically, the confidence output layer receives the defect location, the defect category, and the defect severity from the multi-task learning layer, integrates these three outputs through a fully connected layer, and generates a comprehensive confidence score. This score represents the confidence of the model in the overall defect prediction, including the comprehensive confidence in the defect location, the defect category, and the defect severity. In the model structure, the output results from the three branches (location, category, severity) are spliced together and used as input to the fully connected layer. A value is output through the fully connected layer, representing the comprehensive confidence. Through a sigmoid activation function, the output is converted into a probability value between 0 and 1 as the final confidence score.

[0153] Optionally, the specific structures of the multi-task learning layer and the confidence output layer of the precision detection model for sheet surface defects are the same as those of the multi-task learning layer and the confidence output layer in the magnetic particle defect detection and recognition model.

[0154] As a feasible implementation in this embodiment, the magnetic particle defect detection and recognition model is as follows: First, the input layer receives image input and electromagnetic field data input. The input size of the image is M×H×W×C (C is 15), and the input dimension of the electromagnetic field data is K, representing the electromagnetic field response data at different time steps. The feature extraction layer is divided into an image feature extraction layer and an electromagnetic field feature extraction layer. The image feature extraction layer extracts features from the image data through three convolutional layers, using 64, 128, and 256 convolutional kernels respectively, with a convolutional kernel size of 3x3, and applying batch normalization and max pooling operations after each convolutional layer. The electromagnetic field feature extraction layer compresses and extracts features from the electromagnetic field data through two fully connected layers with 256 and 128 neurons respectively. Subsequently, the model enters the feature fusion layer, where the weighted feature fusion layer weights the image features and the electromagnetic field features, adjusts the weights of the two modal data, generates fused features, and further weights them according to the importance of the features through the attention mechanism layer to ensure that key features occupy a larger proportion during the detection process. The fused features are processed through a fully connected layer (512 neurons) to form more representative global features. The global feature extraction layer further compresses the feature dimension through global average pooling and extracts global features through a fully connected layer with 512 neurons.

[0155] The multi-task learning layer is responsible for predicting the defect location, category, and severity. The defect location branch processes the features step by step through three fully connected layers (with 512, 256, and 128 neurons respectively), and finally outputs 4 values representing the bounding box coordinates of the defect. The defect category branch outputs the prediction results of the defect category through the same three-layer fully connected layer. There are a total of 3 categories, and the output is 3 nodes, representing crack, porosity, and scratch respectively, with the activation function being Softmax. The defect severity branch also outputs 3 nodes representing minor, medium, and severe defects through three fully connected layers, and the activation function is also Softmax. Finally, the confidence output layer comprehensively processes the results from the defect location, defect category, and defect severity branches through a fully connected layer and outputs the confidence score for each detection result.

[0156] Refer to Figure 1 S80 in, correct the first defect detection result according to the second defect detection result to generate a comprehensive result. Integrate the detection information of the two stages to generate a comprehensive result. This result will contain more accurate defect location, category, and severity, providing a basis for subsequent quality control or repair.

[0157] In this embodiment, the preprocessed magnetic particle covered image is input into the trained magnetic particle defect detection and recognition model to obtain multiple groups of detection results (each group includes the detection results of 100 metal sheets 1 to be detected). The detection results of multiple groups corresponding to one image (different combinations of illumination angles) are integrated. Through the integration steps, position correction, category correction, and severity correction are completed in sequence to generate a comprehensive detection result (the second defect detection result). The integration steps are the same as those of the first defect detection result.

[0158] Through the comprehensive processing of multi-modal input data, the magnetic particle defect detection and recognition model improves the comprehensiveness and accuracy of defect detection. The input layer of the model simultaneously receives image data and electromagnetic field data, and uses these two types of information with different dimensions to make up for the limitations of a single data source. The image feature extraction layer and the electromagnetic field feature extraction layer extract visual information and physical information respectively. The feature fusion layer organically combines the two features through weighted fusion and attention mechanism, effectively allocating the weights of different modal information, so as to enhance the model's comprehensive understanding of defects. The global feature extraction layer converts these fused features into more expressive global features through global average pooling, laying a foundation for subsequent multi-task learning. Through the multi-task learning layer, the model can simultaneously output the specific position, category, and severity of the defect, ensuring the comprehensiveness of detection and multi-dimensional analysis. Finally, the confidence output layer generates a confidence score for each detection result, further improving the reliability and accuracy of the detection result. Through multi-modal feature fusion, multi-task learning, and confidence scoring, the overall design enables the model to achieve high-precision and low-error defect recognition in complex detection environments, effectively improving the robustness and adaptability of the detection system.

[0159] In this embodiment, when generating the comprehensive result, the defect position, category, and severity are quantitatively corrected based on the detection information of two stages. First, the defect position correction is completed by calculating the defect position offset between the first stage (surface image detection) and the second stage (magnetic particle detection). If the Euclidean distance between the centers of the two positions is less than the preset threshold, it is considered similar and the result of the first stage is retained; if it is greater than or equal to the preset threshold, the second stage position (the second defect position) is adopted because the accuracy of magnetic particle detection is higher.

[0160] The defect category correction is based on the consistency of the detection results of the two stages. If the categories of the two stages are the same, the category is directly retained; if they are different, it is corrected according to the magnetic particle detection result of the second stage because magnetic particle detection can more accurately distinguish defect types. In the case of inconsistent categories, the result of the second stage is trusted preferentially.

[0161] The defect severity correction is the same as the defect category correction.

[0162] Combined with magnetic particle testing, it can effectively alleviate the deficiency of computer vision-based surface defect detection technology in identifying micro-defects. Magnetic particle testing uses the change of magnetic field to detect the magnetic leakage effect generated by defects such as micro-cracks and pores, and can reveal fine cracks and near-surface defects that are difficult to capture by computer vision. By applying magnetic particle testing to the suspected defect areas identified by visual inspection, the detection accuracy can be further improved, and more precise positioning and analysis of micro-defects can be achieved. Therefore, combining magnetic particle testing can, to a certain extent, make up for the limitations of visual inspection in identifying micro and complex defects and provide more comprehensive detection results.

[0163] The precise detection method for surface defects of metal sheets provided by the present invention can greatly improve the accuracy and reliability of defect detection through multi-angle image acquisition by combining industrial cameras, preliminary defect detection by deep learning models, and precise detection under the action of fluorescent magnetic powder spraying and magnetic field. Compared with traditional single detection methods, this method reduces the false detection rate and missed detection rate through multiple verification steps, and is especially suitable for dealing with complex surface defects. In addition, by correcting the preliminary detection results through magnetic particle testing, the accuracy of the final detection results is ensured, and magnetic powder can be accurately applied according to the severity of the defects, realizing more efficient resource utilization. This comprehensive detection method significantly improves the efficiency and accuracy of the quality inspection of metal sheets at the time of factory shipment and enhances the overall quality control level of products.

[0164] Embodiment 2

[0165] Factory B applies a precise detection method and system for surface defects of metal sheets for the quality inspection of metal sheets at the time of factory shipment. A precise detection method for surface defects of metal sheets applied by it is the same as the method in Embodiment 1, refer to Figure 1 .. The execution of this method is controlled by a precise detection system for surface defects of metal sheets.

[0166] In this embodiment, the metal sheet to be inspected is a rectangular uncoated low-alloy steel sheet with the same size produced in the same batch.

[0167] A precise detection system for surface defects of metal sheets, as Figure 5 shown, includes:

[0168] A metal sheet to be inspected transmission module, used to extract a batch of metal sheets from the ferromagnetic metal sheets that have completed production to obtain N metal sheets to be inspected; the metal sheets to be inspected are the ferromagnetic metal sheets produced in the same batch; move the N metal sheets to be inspected to the designated detection position in sequence through a transmission device;

[0169] An image acquisition module, used to acquire images of the surface of the metal sheet to be inspected at the designated detection position through a first industrial camera to obtain M groups of initial images;

[0170] The sheet surface image preprocessing module is used to preprocess the M groups of the initial images to obtain M groups of preprocessed images;

[0171] The surface defect detection and recognition module is used to input the M groups of the preprocessed images into the trained precise detection model for sheet surface defects to obtain the first defect detection result; the first defect detection result includes the first defect position, the first defect category, and the first defect severity;

[0172] The magnetic powder spraying module is used to calculate the amount of magnetic powder to be sprayed on each defect area according to the first defect detection result, uniformly spray fluorescent magnetic powder on the defect area according to the first defect position corresponding to the defect area and the corresponding amount of magnetic powder to be sprayed, and simultaneously apply a magnetic field to the defect area;

[0173] The fluorescence imaging module is used to use a second industrial camera to take pictures of the defect area under ultraviolet irradiation to obtain a magnetic powder coverage image;

[0174] The magnetic powder image preprocessing module is used to preprocess the magnetic powder coverage image;

[0175] The magnetic powder detection and recognition module is used to input the preprocessed magnetic powder coverage image into the trained magnetic powder defect detection and recognition model to obtain the second defect detection result; the second defect detection result includes the second defect position, the second defect category, and the second defect severity category;

[0176] The result correction module is used to correct the first defect detection result according to the second defect detection result to generate a comprehensive result.

[0177] In this embodiment, the effects of a precise detection method for metal sheet surface defects (Method A) proposed in the present invention, a traditional surface defect detection method (Method B), and a method of using magnetic particle inspection alone (Method C) in the detection of metal sheet surface defects are compared. The experiments evaluate the performance of the three methods in terms of detection accuracy, detection time, false detection rate, and missed detection rate.

[0178] Method B is based on traditional image processing techniques. First, the Sobel edge detection is used to identify the possible defect areas in the image, and then the Otsu adaptive threshold segmentation algorithm is applied to separate the defect areas from the background. The image is further optimized through morphological processing, and finally, the classifier classifies the defects according to geometric features.

[0179] Method C first sprays fluorescent magnetic powder on the surface of the metal sheet and applies a magnetic field to concentrate the magnetic powder at the defects. Subsequently, an industrial camera is used to take a fluorescent image of the defect area under ultraviolet light. The fluorescent area is extracted by color threshold segmentation, and a contour detection algorithm is used to identify the shape and size of the defects. Finally, based on the geometric features of the defects, a classification algorithm is used to classify the defects.

[0180] The experimental objects are ferromagnetic metal sheets produced in the same batch (N = 100 pieces), and each sheet may have various types of surface defects, including cracks, pores, and scratches, etc. The types and distributions of the defects are random.

[0181] Table 3. Data table of comparative experiments

[0182] Method Detection Accuracy (%) Detection Time (minutes) False Detection Rate (%) Missed Detection Rate (%) Method A 98.5 60 1 0.5 Method B 85 45 10 8 Method C 92 120 2 5

[0183] As shown in Table 3, the results show that Method A performs best in terms of detection accuracy, reaching 98.5%, and at the same time has the lowest false detection rate (1%) and missed detection rate (0.5%). Although the detection time is 60 minutes, it is still the detection method with the best comprehensive performance. In contrast, although Method B has the shortest detection time, only 45 minutes, its detection accuracy is relatively low (85%), and the false detection rate and missed detection rate are significantly higher than those of other methods, indicating that traditional image processing techniques have deficiencies in terms of accuracy. Method C shows relatively high accuracy (92%) and a relatively low false detection rate (2%), but due to the complex magnetic powder detection steps, the detection time is the longest, 120 minutes. Generally speaking, Method A has obvious advantages in detection accuracy and stability.

[0184] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for accurately detecting surface defects of metal sheets, characterized in that: include: A batch of metal plates are selected from the ferromagnetic metal plates produced to obtain N metal plates to be inspected; the metal plates to be inspected are the ferromagnetic metal plates produced in the same batch; Move N metal sheets to be inspected to designated inspection positions in sequence through a transmission device; collect images of the surfaces of the metal sheets to be inspected at the designated inspection positions through a first industrial camera to obtain M groups of initial images; Preprocessing the M groups of initial images to obtain M groups of preprocessed images; inputting the M groups of preprocessed images into a trained plate surface defect precision detection model to obtain a first defect detection result; The first defect detection result includes a first defect location, a first defect category, and a first defect severity; Calculate the amount of magnetic powder to be sprinkled on each defect area according to the first defect detection result, evenly spray fluorescent magnetic powder on the defect area according to the first defect position and the corresponding amount of magnetic powder to be sprinkled corresponding to the defect area, and simultaneously apply a magnetic field to the defect area; Using a second industrial camera to photograph the defective area under ultraviolet irradiation conditions to obtain a magnetic powder coverage image; inputting the preprocessed magnetic powder coverage image into a trained magnetic powder defect detection and recognition model to obtain a second defect detection result; The second defect detection result includes a second defect position, a second defect category, and a second defect severity category; The first defect detection result is corrected according to the second defect detection result to generate a comprehensive result.

2. A metal sheet surface defect precision detection method according to claim 1, characterized in that: Moving N metal sheets to be inspected to the designated inspection position in sequence through the transmission device includes: transporting the metal sheets to be inspected from the loading area to the conveyor belt of the transmission device through an automatic loading and positioning device, so that each metal sheet to be inspected moves to the designated inspection position when the conveyor belt stops; the automatic loading and positioning device includes an automatic loading device and a positioning device, the automatic loading device is used to continuously transport the metal sheets to be inspected to the conveyor belt; the positioning device includes a laser projection device, the first industrial camera, an analysis system and a feedback system, the designated inspection position is located at a designated position on the conveyor belt, and the laser projection device The device is installed just above the designated detection position; a virtual fixed frame with the same size as the metal plate to be inspected is projected onto the surface of the conveyor belt by the laser projection device; the first industrial camera is used to capture the image at the designated detection position when the conveyor belt stops conveying, so as to obtain the image to be inspected; the analysis system is used to detect whether the edge of the metal plate to be inspected coincides with the virtual fixed frame according to the image to be inspected, so as to obtain an offset result; when the offset result is yes, an adjustment parameter is calculated; and the feedback system is used to automatically adjust the conveyor belt according to the adjustment parameter when the offset result is yes, so as to make the metal plate to be inspected coincide with the edge of the virtual fixed frame.

3. The method for accurately detecting surface defects of metal sheets according to claim 1, characterized in that: The first industrial camera is used to collect images of the surface of the metal plate to be inspected, and the M groups of initial images are obtained, which are specifically: Under different acquisition angles and lighting conditions, M groups of the initial images are acquired through the first industrial camera, and each group of the initial images corresponds to a different combination of angle parameters and lighting parameters.

4. A metal sheet surface defect precision detection method according to claim 1, characterized in that: The training process of the plate surface defect precision detection model includes the following steps: Data preparation step: collecting metal sheet surface images from different angles and lighting conditions to form M groups of input images; each group of input images is annotated to obtain an annotated input image, wherein the annotated input image contains label information of defect location, defect category, and defect severity; Data preprocessing step: performing data preprocessing on the M groups of labeled input images, including normalization and data enhancement, to obtain preprocessed labeled images; Data division step: dividing the preprocessed labeled images into a first training set and a first validation set; Training step: inputting the first training set into the plate surface defect precision detection model, and using the defect position, defect category and defect severity in the label information to perform supervised learning; during the training process, calculating the position error, classification error and severity error according to the defect position, defect category and defect severity in the label information; calculating the loss function based on the position error, classification error and severity error, updating the model weight parameters through the back propagation algorithm, and obtaining the first plate surface defect precision detection model; Verification step: evaluating the performance of the first plate surface defect precision detection model through the first verification set, calculating the loss function of the first verification set, and if the loss function value does not meet the preset convergence condition, adjusting the hyperparameters of the first plate surface defect precision detection model; The training step and the verification step are repeated until the loss function of the first verification set reaches the preset convergence condition, thereby obtaining the plate surface defect precision detection model.

5. The method for accurately detecting surface defects of metal sheets according to claim 1, characterized in that: The plate surface defect precision detection model includes: A multi-channel input layer, used to receive M sets of input images from different angles and lighting conditions; A feature extraction layer, comprising a plurality of convolutional layers and a plurality of batch normalization layers, for generating a feature map; the convolutional layer is used to extract image features, and the batch normalization layer is used to normalize the output of each of the convolutional layers; A multi-scale feature extraction layer, which uses convolution kernels of different sizes to perform multi-scale processing on the feature map to extract defect features at different scales and generate a multi-scale feature map; A global feature extraction layer, comprising a global average pooling layer and a first fully connected layer; the global average pooling layer performs global information aggregation processing on the multi-scale feature map to obtain global features; and the first fully connected layer converts the global features into global high-dimensional features; a multi-task learning layer, comprising a defect location branch, a defect category branch and a defect severity branch, wherein the defect location branch is used to generate a defect location according to the global high-dimensional feature; the defect category branch is used to generate a defect category according to the global high-dimensional feature; and the defect severity branch is used to generate a defect severity according to the global high-dimensional feature; The confidence output layer is used to comprehensively process the defect location, defect category and defect severity information, generate a comprehensive detection result, and output a confidence score for each of the comprehensive detection results.

6. A metal sheet surface defect precision detection method according to claim 1, characterized in that: Inputting the M groups of preprocessed images into the trained plate surface defect precision detection model to obtain the first defect detection result includes: Inputting the M groups of preprocessed images into the trained plate surface defect precision detection model, respectively obtaining M groups of defect detection results, each group of defect detection results including defect location, defect category, defect severity and detection confidence; The M groups of defect detection results are integrated into the first defect detection result through confidence screening, position alignment, defect category weighted voting and severity weighted averaging; the first defect detection result includes the final defect position, final defect category and final defect severity of each metal plate to be inspected.

7. The method for accurately detecting surface defects of metal sheets according to claim 1, characterized in that: Calculate the amount of magnetic powder to be sprinkled in each defect area according to the first defect detection result, evenly spray fluorescent magnetic powder on the defect area according to the first defect position and the corresponding amount of magnetic powder to be sprinkled corresponding to the defect area, and simultaneously apply a magnetic field to the defect area as follows: The defective area is uniformly sprayed with fluorescent magnetic powder by a magnetic powder spraying device, and a magnetic field is applied simultaneously; the magnetic powder spraying device comprises: A spray head, used for automatically adjusting the opening and closing degree of the nozzle and the spraying angle according to the defect area, so that the fluorescent magnetic powder evenly covers the defect area; A mechanical arm, used for supporting and moving the spray head so that the spray head moves to the defective area; A magnetic powder storage and conveying system, comprising a magnetic powder storage tank and a conveying pipeline, wherein the magnetic powder storage tank is provided with a weighing sensor, the weighing sensor is used to monitor the amount of remaining magnetic powder in real time, and the conveying pipeline is used to convey the fluorescent magnetic powder from the magnetic powder storage tank to the spray head; An electromagnet module is used to apply a magnetic field synchronously with the spraying of fluorescent magnetic powder; The magnetic powder spraying device also includes: a calculation and analysis module, which is used to calculate the amount of magnetic powder to be sprayed in each defect area according to the first defect detection result; obtain a spraying density, a spraying number and a spraying time according to the first defect detection result, and control the operating parameters of the spray head, the robotic arm and the electromagnet module according to the spraying density, the spraying number, the spraying time and the amount of magnetic powder to be sprayed.

8. The method for accurately detecting surface defects of metal sheets according to claim 1, characterized in that: The training process of the magnetic particle defect detection and recognition model includes the following steps: The surface images of the metal sheet covered with fluorescent magnetic powder are collected from different angles to form multiple groups of magnetic powder input images, and the electromagnetic field response data during and after the fluorescent magnetic powder spraying are collected at the same time; the multiple groups of the magnetic powder input images are annotated, and the annotation content includes defect location, defect category and defect severity, to obtain annotated magnetic powder images, and the annotated magnetic powder images are respectively combined with the corresponding electromagnetic field response data to obtain multimodal data; the multimodal data are preprocessed to obtain preprocessed multimodal data; the preprocessed multimodal data is divided into a second training set and a second verification set; and the trained magnetic powder defect detection and recognition model is obtained according to the second training set and the second verification set.

9. The method for accurately detecting surface defects of metal sheets according to claim 1, characterized in that: The magnetic particle defect detection and recognition model includes: An input layer, used to receive multimodal input data; the multimodal input data includes image input and electromagnetic field data input; The feature extraction layer includes an image feature extraction layer and an electromagnetic field feature extraction layer; the image feature extraction layer includes a plurality of convolutional layers for extracting features of the image input and generating image features; the electromagnetic field feature extraction layer is used to extract features of the electromagnetic field data input and generate electromagnetic field features; The feature fusion layer includes a weighted feature fusion layer and an attention mechanism layer, and is used to fuse the image features and the electromagnetic field features; the weighted feature fusion layer adjusts the weight ratio of the image features and the electromagnetic field features by learning the weights of different modal input data in the multimodal input data to generate fused features; the attention mechanism layer assigns weights to different features of the fused features according to feature importance; A global feature extraction layer, including a global average pooling layer, for converting the image features and the electromagnetic field features into global features; The multi-task learning layer includes a defect location branch, a defect category branch, and a defect severity branch; the defect location branch is used to generate a defect location based on the global feature; the defect category branch is used to generate a defect category based on the global feature; the defect severity branch is used to generate a severity based on the global feature; The confidence output layer receives the defect location, defect category and defect severity from the multi-task learning layer, generates a comprehensive detection result, and generates a confidence score for each comprehensive detection result through a fully connected layer.

10. A metal sheet surface defect precision detection system, characterized in that: include: The sheet material transmission module to be inspected is used to extract a batch of metal sheets from the ferromagnetic metal sheets that have been produced to obtain N metal sheets to be inspected; the metal sheets to be inspected are the ferromagnetic metal sheets produced in the same batch; and the N metal sheets to be inspected are sequentially moved to the designated inspection position through the transmission device; An image acquisition module, used for acquiring images of the surface of the metal sheet to be inspected at the designated inspection position by a first industrial camera to obtain M groups of initial images; A plate surface image preprocessing module is used to preprocess the M groups of initial images to obtain M groups of preprocessed images; A surface defect detection and recognition module, used for inputting the M groups of pre-processed images into a trained plate surface defect precision detection model to obtain a first defect detection result; The first defect detection result includes a first defect location, a first defect category, and a first defect severity; a magnetic powder spraying module, configured to calculate the amount of magnetic powder to be sprayed on each defective area according to the first defect detection result, uniformly spray fluorescent magnetic powder on the defective area according to the first defect position and the corresponding amount of magnetic powder to be sprayed corresponding to the defective area, and simultaneously apply a magnetic field to the defective area; A fluorescent imaging module, used to use a second industrial camera to photograph the defective area under ultraviolet irradiation conditions to obtain a magnetic powder coverage image; A magnetic powder image preprocessing module, used for preprocessing the magnetic powder coverage image; A magnetic particle detection and recognition module, used for inputting the preprocessed magnetic particle coverage image into a trained magnetic particle defect detection and recognition model to obtain a second defect detection result; The second defect detection result includes a second defect position, a second defect category, and a second defect severity category; The result correction module is used to correct the first defect detection result according to the second defect detection result to generate a comprehensive result.

Images