Steel sheet defect detection method, storage medium, electronic equipment and device

Abstract

The invention relates to the technical field of printers, in particular to a steel sheet defect detection method, a storage medium, electronic equipment and device.The steel sheet defect detection method comprises the steps that firstly, an image of a to-be-detected steel sheet is obtained, a to-be-detected image is obtained, then the to-be-detected image is input into a defect detection model for defect type recognition, and the corresponding defect type is obtained; the training sample of the defect detection model comprises one or more of the following defect types of marking images: fillet convex points, fillet concave points, fillet cracks, fillet size non-conforming to the standard and deformation. By acquiring the image of the to-be-detected steel sheet and inputting the image into the defect detection model for identification, automation of steel sheet defect detection is achieved, tedious steps of manual detection are avoided, defect detection of a large number of steel sheets can be rapidly achieved, and therefore the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of printer technology, and in particular to a steel sheet defect detection method, storage medium, electronic equipment and device. Background Art

[0002] In modern office environments, laser printers are widely used for their high efficiency and high-quality print output. One of the core components of a laser printer is the toner cartridge (photosensitive drum), which is responsible for forming an electrostatic latent image and transferring the image to paper through toner. In order to ensure the print quality and extend the service life of the toner cartridge, it is necessary to regularly remove excess toner and impurities on the surface of the photosensitive drum. This task is mainly completed by the toner cartridge scraper, especially the steel scraper, which is widely used due to its good wear resistance and cleaning effect.

[0003] The steel blade scraper is usually installed in the drum assembly, and the steel blade on it is in close contact with the surface of the photosensitive drum. When the photosensitive drum rotates, the steel blade can effectively scrape off excess toner, ensuring that the surface of the photosensitive drum remains clean, thereby ensuring that every print can produce a clear, flawless image. In addition, the steel blade can also help remove tiny particles and other contaminants on the surface of the photosensitive drum, reduce wear on the photosensitive layer, and extend the service life of the drum. Therefore, the quality of the steel blade directly affects the performance and reliability of the entire printer.

[0004] The existing production process of steel sheets is as follows: 1) Material selection - choose suitable stainless steel materials as steel sheets; 2) Cutting and forming: use precision shearing or laser cutting technology to cut stainless steel sheets into the required size and shape, and use stamping, bending and other processes to form the steel sheets into the structure required by the design; 3) Surface treatment: clean the steel sheets, remove impurities such as oil and oxides, make the surface of the steel sheets smooth and flat by grinding or polishing, and apply rust inhibitors or wear-resistant coatings if necessary; 4) Quality inspection: because the quality of the steel sheet directly affects the performance and life of the steel sheet scraper, which in turn leads to printing quality problems and even damage to the toner cartridge, the quality of the steel sheet needs to be inspected to select defective products. The existing quality inspection relies on manual visual inspection to check whether there are defects in the appearance of the steel sheet. Although this method is accurate, it is inefficient, time-consuming, and easily affected by human factors. In addition, the cost of manual inspection is high, which is not conducive to large-scale production and quality control.

[0005] Therefore, how to improve the efficiency of defect detection of steel sheets and realize automated and high-precision detection is a technical problem that technicians currently need to solve. Summary of the invention

[0006] The present invention provides a steel sheet defect detection method, a storage medium, an electronic device and a device, which are used to solve the technical problem of low detection efficiency caused by the detection method of steel sheet defects through manual visual inspection in the prior art.

[0007] A first aspect of the present invention provides a steel sheet defect detection method, comprising:

[0008] S1: Acquire the image of the steel sheet to be inspected to obtain the image to be inspected;

[0009] S2: The image to be inspected is input into a defect detection model to identify the defect type and obtain the corresponding defect type. The training samples of the defect detection model include annotated images of one or more of the following defect types: rounded corner convex points, rounded corner concave points, rounded corner cracks, rounded corner dimensions that do not meet the standards, and deformation.

[0010] In a first possible steel sheet defect detection method of the first aspect, before S1, the method further includes:

[0011] S01: acquiring images of area A and area B of the steel sheet to obtain image A and image B, where area A and area B are symmetrical about the width axis of the steel sheet;

[0012] S02: Compare and analyze image A and image B to obtain the difference K between image A and image B;

[0013] S03: Determine whether K meets the preset standard;

[0014] S04: If it meets the requirements, S01 to S03 are executed cyclically, and the area A corresponding to each cycle does not overlap with the union of the previous area A, and the area B corresponding to each cycle does not overlap with the union of the previous area B, until the union of all areas A and B coincides with the welding surface, it is determined to be qualified, and the steel sheet is sent to the processing station to execute the next production process;

[0015] S05: If not, it is determined that there is a defect and step S1 is executed.

[0016] In combination with the first possible implementation of the steel sheet defect detection method of the first aspect, in a second possible implementation of the steel sheet defect detection method of the first aspect, S05 includes:

[0017] S051: If not, it is determined to be defective;

[0018] S052: stitching image A and image B according to the outline of the steel sheet, the order and direction of image acquisition, to obtain a stitched image C;

[0019] S053: Acquire image D, where image D is an image corresponding to the area on the steel sheet that is not covered by area A and area B;

[0020] S1 includes: splicing image C and image D to obtain an image to be inspected.

[0021] In combination with the first possible implementation of the steel sheet defect detection method of the first aspect, in a third possible implementation of the steel sheet defect detection method of the first aspect, S1 includes:

[0022] S11: Acquire the image of the steel sheet to be inspected to obtain the image to be inspected;

[0023] S12: determining preliminary defect information of the image to be inspected according to the difference K, where the preliminary defect information includes a defect area and / or a defect edge;

[0024] S13: preprocessing and marking the image to be inspected according to the preliminary defect information;

[0025] The defect detection model is provided with an attention mechanism, and the preliminary defect information serves as the input of the attention mechanism.

[0026] In combination with the first possible implementation of the steel sheet defect detection method of the first aspect, in a fourth possible implementation of the steel sheet defect detection method of the first aspect, the acquisition directions of image A and image B are opposite or opposite.

[0027] In combination with the steel sheet defect detection method of the first aspect, the first possible implementation of the steel sheet defect detection method of the first aspect, the second possible implementation of the steel sheet defect detection method of the first aspect, the third possible implementation of the steel sheet defect detection method of the first aspect, or the fourth possible implementation of the steel sheet defect detection method of the first aspect, in the fifth possible implementation of the steel sheet defect detection method of the first aspect, S02 includes:

[0028] Image A and image B are compared and analyzed by using any one or more of pixel level comparison, structural similarity index, feature matching and histogram comparison to obtain a difference K between image A and image B.

[0029] In combination with the fifth possible implementation of the steel sheet defect detection method of the first aspect, in a sixth possible implementation of the steel sheet defect detection method of the first aspect, when the structural similarity index is used, S02 includes:

[0030] S021: Convert image A and image B into grayscale images to obtain grayscale image A and grayscale image B;

[0031] S022: For the corresponding positions of the grayscale image A and the grayscale image B, use the selected window sliding to calculate the local SSIM value;

[0032] S023: Calculate the average of all local SSIM values ​​to obtain the global SSIM value.

[0033] A second aspect of the present invention provides an electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any possible steel sheet defect detection method provided in the first aspect.

[0034] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any possible steel sheet defect detection method provided in the first aspect.

[0035] A fourth aspect of the present invention provides a steel sheet defect detection device, comprising:

[0036] A positioning component, used to fix the steel sheet at the image acquisition station;

[0037] An image acquisition component, used for acquiring images of steel sheets;

[0038] Image analysis component, used to analyze and process images.

[0039] It can be seen from the above technical solutions that the present invention has the following advantages:

[0040] The steel sheet defect detection method provided by the present invention realizes the automation of steel sheet defect detection by acquiring the image of the steel sheet to be detected and inputting the defect detection model for identification. Automated detection avoids the cumbersome steps of manual detection and can quickly realize the defect detection of a large number of steel sheets, thereby improving the detection efficiency.

[0041] The defect detection model has been trained to identify a variety of defect types, including rounded corner convexities, rounded corner concave points, rounded corner cracks, rounded corner dimensions that do not meet standards, and deformations. Compared with manual inspection, the defect detection model has been trained with a large number of samples and can identify defects more accurately and reduce the misjudgment rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0043] Figure 1 A schematic flow chart of a steel sheet defect detection method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The embodiments of the present invention provide a steel sheet defect detection method, storage medium, electronic device and apparatus, which are used to solve the technical problem that the detection method of steel sheet defects through manual visual inspection in the prior art leads to low detection efficiency.

[0045] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] In the description of the embodiments of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0047] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a replaceable connection, or an integral connection, it can be a mechanical connection, it can be an electrical connection, it can be a direct connection, it can be indirectly connected through an intermediate medium, and it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0048] The existing quality inspection relies on manual visual inspection of the steel sheet’s appearance for defects. Although this method is accurate, it is inefficient, time-consuming, and easily affected by human factors. In addition, manual inspection is costly and is not conducive to large-scale production and quality control.

[0049] Embodiment 1

[0050] See also Figure 1 The steel sheet defect detection method provided by the embodiment of the present invention comprises:

[0051] S1: Acquire the image of the steel sheet to be inspected to obtain the image to be inspected;

[0052] Specifically, an industrial camera or a high-resolution camera is used to capture images of the steel sheet under appropriate lighting conditions. The captured images are transmitted to an image processing system for necessary preprocessing, such as image cropping, noise removal, etc., to obtain an image to be inspected. Exemplary: Assume that an industrial camera with a resolution of 1920x1080 is used to capture images of the steel sheet under uniform white light illumination to obtain an original image. The original image is transmitted to a computer, cropped using image processing software, and the background area is removed to obtain an image to be inspected with a size of 1000x800.

[0053] S2: Input the image to be inspected into a defect detection model to identify the defect type and obtain the corresponding defect type. The training samples of the defect detection model include annotated images of one or more of the following defect types: rounded corner convex points, rounded corner concave points, rounded corner cracks, rounded corner dimensions that do not meet the standards, and deformation.

[0054] Specifically, the step includes:

[0055] Collect and prepare training samples:

[0056] Collect a large number of steel sheet images containing various defect types, such as rounded corner convex points, rounded corner concave points, rounded corner cracks, rounded corner dimensions that do not meet standards, and deformation (including small edge deformation and plane deformation). Annotate the collected images, clarify the location, type, and other information of the defects in each image, and obtain labeled defect type images.

[0057] Build a neural network model: Select a suitable neural network architecture, such as convolutional neural network (CNN), deep residual network (ResNet), dense network (DenseNet), etc. Adjust the network structure according to specific needs, such as increasing or decreasing the number of convolutional layers, adjusting the size of the convolution kernel, introducing skip connections, etc.

[0058] Data preprocessing:

[0059] Preprocessing operations are performed on the training samples, such as image scaling (adjusting the image size to the model input size), normalization (scaling the pixel values ​​to the range of [0, 1] or [-1, 1]), data augmentation (random rotation, flipping, cropping, adding noise, etc.), etc., to improve the generalization ability of the model.

[0060] Model training:

[0061] The preprocessed training samples are input into the neural network model and forward propagation is performed to obtain the prediction results. The loss value between the prediction result and the true label (such as cross entropy loss, mean square error loss, etc.) is calculated. The back propagation algorithm calculates the gradient according to the loss value and updates the network parameters (such as weights, biases, etc.) to minimize the loss value. Repeat the above process until the performance of the model on the validation set reaches the expected value (for example, the accuracy rate reaches more than 95%).

[0062] Model evaluation and optimization:

[0063] Use an independent test set to evaluate the trained model and test the model's performance indicators such as accuracy, precision, recall, F1 score, etc. Optimize the model based on the evaluation results, such as adjusting hyperparameters (learning rate, batch size, etc.), increasing training data, and using regularization techniques (such as dropout, batch normalization, etc.).

[0064] Model saving and application:

[0065] Save the trained model for use in subsequent defect detection tasks. Input the image to be inspected into the trained defect detection model, and the model outputs information such as defect type, location, and confidence.

[0066] Exemplary: Assume that a convolutional neural network (CNN) is used as a defect detection model: 10,000 images of steel sheets containing various defect types are collected, and each image is annotated to obtain a labeled defect type image. The image size is adjusted to 224x224, and the image is normalized. Data enhancement techniques are used, such as random rotation of 0-360 degrees, random flipping, random cropping, etc., to increase the diversity of training data. A CNN model with 5 convolutional layers and 2 fully connected layers is constructed, and the ReLU activation function and softmax classifier are used. The preprocessed training samples are input into the CNN model, and forward propagation and back propagation are performed to update the network parameters. After 50 epochs of training, the accuracy of the model on the validation set reached 96%. The model was evaluated using the test set, and the accuracy, precision, recall and F1 scores were 95.5%, 94.8%, 96.2%, and 95.5%. The trained CNN model is saved and applied to the steel sheet defect detection task. The image to be inspected is input into the CNN model, and the model outputs the defect type as "rounded corner crack" with a confidence level of 0.85.

[0067] The beneficial effects of this embodiment include:

[0068] ① By acquiring the image of the steel sheet to be inspected and inputting the defect detection model for identification, the automation of steel sheet defect detection is realized. Automated detection avoids the tedious steps of manual detection and can quickly realize the defect detection of a large number of steel sheets, thereby improving the detection efficiency.

[0069] ② The defect detection model has been trained to identify various types of defects, including rounded corner convex points, rounded corner concave points, rounded corner cracks, rounded corner dimensions that do not meet standards, and deformation. Compared with manual inspection, the defect detection model has been trained with a large number of samples and can identify defects more accurately and reduce the misjudgment rate.

[0070] ③The automated detection system can replace a large amount of manual detection work, reduce manpower input, and reduce labor costs.

[0071] Although the above detection method can realize the defect detection of steel sheets, realize the classification of qualified products and defective products and further classification of defective products, in the actual production process, the number of qualified steel sheets is much greater than the number of defective steel sheets. Therefore, if all steel sheets are defectively detected according to the above defect detection method, a large number of qualified products need to be subjected to detailed defect detection and analysis, which takes a lot of time and affects the overall production efficiency. In order to improve the overall production efficiency, the following steps are added before step S1, and the preliminary classification of qualified products and defective products is realized through the following steps before detailed defect detection, thereby improving the overall production efficiency.

[0072] S01: acquiring images of area A and area B of the steel sheet, obtaining images A and B, where area A and area B are symmetrical about the width axis of the steel sheet;

[0073] Specifically, if there are no defects in the steel sheet, any two areas symmetrical about the width axis of the steel sheet are highly similar, and the width axis of the steel sheet refers to the symmetry axis on the steel sheet perpendicular to the length direction of the steel sheet. In this step, two high-resolution cameras or other image acquisition devices are used to respectively shoot the area A and area B symmetrical about the width axis on the steel sheet, and obtain image A corresponding to area A and image B corresponding to area B. By acquiring the images of the symmetrical areas, basic data is provided for subsequent comparative analysis. It should be understood that area A and area B can be selected arbitrarily, as long as they are symmetrical about the width axis, such as selecting two areas close to the width axis, or selecting two areas far from the width axis. In addition, by shooting the images of area A and area B by the image acquisition device, the obtained images A and B will contain the background image. In order to avoid the interference of the background image, the existing image segmentation technology can be used to pre-process the initial shooting through coordinate information or pixel information, and the images of area A and area B can be removed. The imaging area of ​​the non-steel sheet can also be masked by designing the corresponding station to make the image of the background part the same everywhere, so that the interference of the background can also be eliminated.

[0074] S02: Compare and analyze image A and image B to obtain the difference K between image A and image B;

[0075] Specifically, in this step, in order to determine how much difference there is between image A and image B, image A and image B are compared and analyzed by an existing image comparison analysis method, so as to obtain the difference between image A and image B. Because different image comparison analysis methods have different comparison indicators, the difference degree K is specifically determined by the image comparison analysis method adopted. For example, if the structural similarity index (SSIM) is adopted, the difference degree is the global SSIM value of image A and image B.

[0076] S03: Determine whether K meets the preset standard;

[0077] Specifically, by setting a preset standard and comparing the difference obtained in S02 with the preset standard, it is determined whether the difference K is within an acceptable range. The preset standard is the same indicator as the difference K, which can be expressed as a preset value or a preset range, and is specifically determined by the image comparison and analysis method used. The preset standard can be set by empirical method, statistical analysis method, and experimental adjustment method, wherein the empirical method is to select a reasonable preset standard based on past empirical data, the statistical analysis method is to perform statistical analysis on a large amount of sample data to find a suitable preset standard, and the experimental adjustment method is to find the optimal preset standard through continuous trial and adjustment in actual application. The preset standard setting should consider the balance between the false alarm rate and the false negative rate, that is, to reduce erroneous similarity judgments and avoid missing truly similar images.

[0078] S04: If it meets the requirements, S01 to S03 are executed cyclically, and the area A corresponding to each cycle does not overlap with the union of the previous area A, and the area B corresponding to each cycle does not overlap with the union of the previous area B, until the union of all areas A and B coincides with the welding surface, it is determined to be qualified, and the steel sheet is sent to the processing station to execute the next production process;

[0079] Specifically, if it meets the requirements, it means that there are no defects in the current area A and area B, but the requirement for the qualified quality of the steel sheet is that there are no defects in the entire welding surface, so the remaining undetected areas of the welding surface should continue to be detected. Each detection of a new area is to repeat steps S01 to S03. If the difference of the newly detected area also meets the preset standard, the next new area will continue to be detected until the union of all detection areas coincides with the welding surface, that is, the entire welding surface has been detected, that is, the entire steel sheet has been detected, and the product is determined to be qualified. The qualified steel sheet is sent to the next processing station through a manipulator or other automated equipment to ensure the continuity of the production line and the quality of the product. If the difference between two symmetrical areas does not meet the preset standard, step S05 is entered. In each cycle, it is necessary to ensure that the current detection area has a part of the undetected area to be detected compared with the previous detection area, that is, the current detection area and the previously detected area may have an intersection, but the area of ​​the intersection must be smaller than the area of ​​the current detection area. It should be noted that when the symmetrical area A and area B have exactly the same defects at the same position, and there are only these two defects on the welding surface, there may be a possibility of missed detection, but the probability of this is extremely low and within the acceptable error range.

[0080] More preferably, because there are high-defect areas, the steel sheet can be divided into high-defect areas and non-defect high-incidence areas. The high-defect areas can be used as the first few areas A and B, and the non-defect high-incidence areas can be used as the subsequent areas A and B. In this way, the detection efficiency of determining whether there are defects can be improved. In addition, the areas of areas A and B in the high-defect areas can be set smaller, and the areas of areas A and B in the non-defect high-incidence areas can be set larger, so that the detection efficiency can be further improved.

[0081] More preferably, in order to reduce the moving distance of the image acquisition device, the acquisition directions of image A and image B can be set to be opposite or opposite. Opposite means that images are gradually acquired from the area close to the width axis toward both ends in the length direction of the steel sheet; opposite means that images are gradually acquired from both ends in the length direction of the steel sheet toward the width axis.

[0082] S05: If not, it is determined that there is a defect and step S1 is executed.

[0083] Specifically, if the difference between two symmetrical areas does not meet the preset standard, it means that there are defects in the two symmetrical areas. For the purpose of improving production efficiency, it is necessary to quickly pick out products with qualified welding quality from all produced products. When it is determined that there are defects in the two symmetrical areas, it is also determined that the corresponding steel sheets are defective, and the defective products are picked out, that is, the classification of qualified products and defective products is completed, and then step S1 is executed to identify the defect type and defect level of the defective products.

[0084] In order to improve the image acquisition efficiency and reduce the burden on the image acquisition device, step S05 is optimized as follows:

[0085] S051: If not, it is determined to be defective;

[0086] Specifically, if the difference between two symmetrical regions does not meet the preset standard, it means that there are defects in the two symmetrical regions.

[0087] S052: stitching image A and image B according to the outline of the steel sheet, the order and direction of image acquisition, to obtain a stitched image C;

[0088] Specifically, in this step, the purpose is to use the image of the detected area of ​​the steel sheet at the joint of the acquired image A and image B. When it is preliminarily determined that there are defects, two symmetrical area images on both sides of the width axis of the steel sheet have been collected, recorded as image A and image B, and preprocessed on image A and image B, including denoising, graying, edge enhancement, etc., to improve the quality of stitching, and use feature point detection algorithms (such as SIFT, SURF, ORB) to detect feature points in image A and image B. Take the area corresponding to image A and image B as an example to illustrate, and find the matching feature point pairs in the two images through the feature descriptor matching algorithm (such as FLANN matcher). Use the RANSAC algorithm to calculate the homography matrix between image A and image B to eliminate the influence of perspective transformation. Image A and image B are spliced ​​according to the homography matrix to form a preliminary complete image, recorded as image C. During the splicing process, image fusion technology (such as multi-band fusion) can be used to reduce the splicing seam and brightness difference.

[0089] S053: Acquire image D, where image D is an image corresponding to the area on the steel sheet that is not covered by area A and area B;

[0090] Specifically, after completing the stitching of S052, the uncovered steel sheet areas in image C are identified, and images of these uncovered areas are collected and recorded as images D. It is ensured that these images have sufficient overlapping areas with image C, and the newly collected image D is preprocessed, including denoising, grayscale, edge enhancement, etc., to improve the stitching quality.

[0091] Accordingly, S1 is optimized as follows: splicing image C and image D to obtain the image to be inspected.

[0092] Specifically, feature point detection and matching algorithms are also used to find matching feature points between image C and image D. The homography matrix between image C and the new image is calculated. Image D is spliced ​​with image C to form the final complete image to be inspected, and image fusion technology is also used to process the splicing seams and brightness differences.

[0093] Compared with directly collecting the complete image again, obtaining the complete steel sheet image to be inspected through image stitching has the following advantages: by collecting the symmetrical area and the remaining area in different areas, parallel processing or fast switching of the collection area can be performed, saving time; there is no need to move the entire steel sheet at once or adjust the position of the collection equipment during the collection process, reducing the time of mechanical movement and adjustment; the area collection can use a lower resolution camera, because the image size of each area is small, and finally a high-resolution complete image is obtained through stitching; reducing the dependence on high-resolution, high-performance equipment, extending the service life of the equipment, and reducing maintenance costs; the image of each area can be optimized and processed separately (such as denoising, enhancement), improving the local image quality, thereby improving the overall quality of the final stitched image. During the stitching process, preliminary defect detection and analysis can be performed on each area to improve the accuracy of detection. It can be easily expanded to larger or more complex steel sheet inspection tasks.

[0094] Through defect detection, products can be quickly classified into qualified products and defective products. The qualified products are then sent to the next production process for processing, and the defective products are sent to the defect detection station for specific defect analysis to obtain the specific defect type. Compared with the traditional method of performing a complete defect analysis at one station and then classifying, qualified products can be selected in a shorter time and sent to the next process for processing. By separating the specific analysis of defects from the main processing flow, the impact of defect analysis on processing speed is avoided, thereby improving the overall processing efficiency.

[0095] After initially determining that the steel sheet has defects, the obtained defect information (such as defect location and type) is used to guide subsequent image stitching and defect type identification, which can significantly reduce the recognition difficulty of large models and improve recognition efficiency and accuracy. Based on this, step S1 is optimized as follows:

[0096] S11: Acquire the image of the steel sheet to be inspected to obtain the image to be inspected;

[0097] S12: determining preliminary defect information of the image to be inspected according to the difference K, where the preliminary defect information includes a defect area and / or a defect edge;

[0098] Specifically, when making a preliminary defect determination, the approximate position of the defect can be located through simple image processing techniques (such as edge detection, threshold segmentation) or rule-based algorithms. For example, in order to determine the specific area that causes the difference K, the pixel difference between image A and image B is calculated to generate a difference map. Each pixel value in the difference map represents the grayscale difference at the corresponding position. The difference map is binarized, and the areas with significant differences are marked as white (or 1), and the areas with insignificant differences are marked as black (or 0). Through connected domain analysis, all connected areas in the difference map are found, and these areas are defective areas. For example, if an abnormal brightness change is detected in the edge area, it may be preliminarily determined to be a defective edge.

[0099] In addition, during the image stitching process, the defective area can be stitched and focused on first. By giving priority to the defective area, the calculation of the non-defective area can be reduced and the processing speed can be improved. Assume that it is initially determined that there is a defect on the left edge of the steel sheet. When stitching images, the area image containing the left edge is stitched first to ensure the image quality and high resolution of the area. In subsequent processing, the left edge area in the stitched image can be used as the focus of analysis to reduce the amount of calculation for other areas.

[0100] S13: preprocessing and marking the image to be inspected according to the preliminary defect information;

[0101] Specifically, according to the preliminary defect information, adjust the image preprocessing parameters. For example, if it is initially determined to be a scratch defect, the edge information of the image can be enhanced; if it is initially determined to be a surface stain, the contrast and brightness can be adjusted. By adjusting the preprocessing parameters, the defect features can be highlighted and the recognition accuracy of the large model can be improved.

[0102] In addition, when stitching images, the defective areas can be processed more finely, such as using high-resolution stitching algorithms or local optimization algorithms to reduce stitching errors and improve image quality. You can also mark the location and type of the defective area on the image before inputting it into the large model to provide additional information to help the model identify it, which can help the large model identify defects faster and more accurately. Or based on the preliminary defect information, the stitched complete image can be cropped and scaled to highlight the defective area. For example, if it is initially determined that the defect is located in a specific area of ​​the image, the area can be cropped and used as the input of the large model. By cropping and scaling, the defective area is highlighted, the size and complexity of the input image are reduced, and the computational efficiency of the model is improved.

[0103] Accordingly, an attention mechanism is introduced into the large model so that the model pays more attention to the defect area during the recognition process. For example, spatial attention or channel attention can be used to guide the model to focus on a specific area. Preliminary defect information can be used as the input of the attention mechanism to guide the model to focus on a specific area. For example, the location information of the preliminary defect is input into the attention mechanism to guide the model to focus on the defect area. For example, using the spatial attention mechanism, an attention module is introduced into the model to generate an attention map to highlight the defect area. Preliminary defect information (such as the defect location) is input into the attention mechanism to guide the model to focus on the defect area. The attention mechanism can make the model pay more attention to the defect area and improve the recognition accuracy. By guiding the model to focus on specific areas, unnecessary calculations can be reduced and the calculation efficiency can be improved.

[0104] Using preliminary defect information to locate defect areas, optimize image preprocessing, optimize large model input, and introduce attention mechanisms can effectively reduce the recognition difficulty of large models and improve the efficiency and accuracy of defect detection. These methods can not only improve image quality and stitching accuracy, but also optimize model input and training processes, thereby achieving more efficient and accurate defect detection.

[0105] In order to improve the accuracy and robustness of the difference calculation, step S02 is optimized as follows: using any one or more of the methods of pixel-level comparison, structural similarity index, feature matching and histogram comparison to compare and analyze image A and image B, and obtaining the difference K between image A and image B, where K is expressed as different indicators depending on the specific method used.

[0106] Specifically, one method among pixel level comparison, structural similarity index, feature matching and histogram comparison may be used alone for comparative analysis, or a combination of multiple methods among pixel level comparison, structural similarity index, feature matching and histogram comparison may be used for comprehensive comparative analysis.

[0107] Pixel-level comparison can be done by using the absolute difference method or the mean square error (MSE). The specific process of the absolute difference method is: calculate the difference between the RGB values ​​(or grayscale values) of the corresponding pixels of image A and image B, and then calculate the average or sum of all the differences as the difference K. The specific process of the mean square error (MSE) can also be used: calculate the average of the sum of the squares of the differences between the RGB values ​​(or grayscale values) of the corresponding pixels of image A and image B as the difference K.

[0108] The specific process of using the structural similarity index (SSIM) is as follows: convert image A and image B into grayscale images, select a window size (e.g. 7x7 pixels), and slide the window across the entire image. At each window position, calculate the local SSIM value. Calculate the average of all local SSIM values ​​to get the global SSIM value.

[0109] Feature matching can use scale-invariant feature transform (SIFT), speeded up robust features (SURF) or oriented FAST and rotated BRIEF (ORB), and the number of matching points or the quality of matching points can be used as the difference K. When based on the number of matching points: extract key points and their descriptors through SIFT, SURF or ORB algorithms, and find the best matching points through BFMatcher or other matching algorithms; use RANSAC (for SIFT / SURF) or ratio test (for ORB) to remove wrong matching points and retain high-quality matching points; use the number of high-quality matching points finally retained as the difference K. The more matching points there are, the higher the similarity between the two images; conversely, the fewer matching points there are, the greater the difference between the two images. When based on the quality of matching points: find the best matching points through BFMatcher or other matching algorithms, and record the distance of each matching point (i.e. the difference between descriptors); use RANSAC (for SIFT / SURF) or ratio test (for ORB) to remove incorrect matching points and retain high-quality matching points; calculate the average distance of all high-quality matching points as the difference degree K. The smaller the matching point distance, the higher the similarity between the two images; conversely, the larger the matching point distance, the greater the difference between the two images.

[0110] Exemplary: When the structural similarity index is used, S02 includes:

[0111] S021: Convert image A and image B into grayscale images to obtain grayscale image A and grayscale image B;

[0112] Specifically, the image A and the image B are converted into grayscale images, and Gaussian blur or other denoising algorithms are applied to remove noise from the grayscale images to obtain grayscale images A and grayscale images B.

[0113] S022: For the corresponding positions of the grayscale image A and the grayscale image B, use the selected window sliding to calculate the local SSIM value;

[0114] Specifically, select a suitable window size (e.g. 7x7 pixels) for the SSIM calculation of the local area, and use the selected window sliding to calculate the local SSIM value for the corresponding position of each pair of symmetric regions. The formula is as follows:

[0115]

[0116] Where: μ x and μ y are the local means of two local regions, and are the local variances of the two local regions, are the covariances of the two local regions, and C 1 and C 2 is a constant used to stabilize the denominator and prevent division by zero errors.

[0117] S023: Calculate the average of all local SSIM values ​​to obtain a global SSIM value, and use the global SSIM value as the difference K.

[0118] Set a suitable SSIM threshold based on experimental or empirical data. For example, if the SSIM values ​​of image A and image B are higher than 0.95, they are considered to be very similar and have no obvious welding defects. If the global SSIM value is lower than the threshold, it means that there are significant differences between the two symmetrical regions and there may be welding defects. The area with lower local SSIM values ​​can be further checked to locate the specific defect position.

[0119] Embodiment 2

[0120] An embodiment of the present invention provides a steel sheet defect detection device, comprising:

[0121] The positioning component is used to fix the steel sheet at the image acquisition station. The steel sheet is fixed by the precise positioning component to ensure that area A and area B are strictly symmetrical each time they are photographed; the image acquisition component is used to capture the image of the steel sheet; the image analysis component is used to analyze and process the image.

[0122] Specifically, the positioning assembly is usually composed of high-precision mechanical fixtures or positioning tables, which can accurately fix the steel sheet to ensure that its position is stable and highly repeatable. Before installation and use, the positioning assembly needs to be calibrated to ensure that it can accurately fix the steel sheet in the predetermined position. The calibration process may include the use of laser alignment, mechanical adjustment and other methods. Sensors (such as photoelectric sensors, position sensors) can be used to assist in positioning to ensure that the position of the steel sheet is accurate. Exemplary: The positioning assembly is a precision mechanical fixture that fixes the steel sheet with two adjustable clamping arms. During installation, the position of the clamping arm is adjusted by the laser alignment system to ensure that the area to be inspected of the steel sheet corresponds to the optical axis of the image acquisition device, and that area A and area B are symmetrical about the width axis of the steel sheet.

[0123] The image acquisition component uses a high-resolution industrial camera or other image acquisition device to ensure that the image quality is high enough to capture subtle defects. In order to obtain a clear image, a suitable light source is required. LED lights or other high-brightness light sources can be used, and diffuse reflectors or ring light sources can be used to evenly illuminate the welding surface. In order to avoid background interference, image segmentation technology or methods of masking non-welding surface areas can be used during image acquisition. For example, a light shield can be designed to expose only part of the welding surface. Exemplary: Use two high-resolution industrial cameras, symmetrically placed on both sides of the steel sheet. Each camera is equipped with a ring LED light source to ensure uniform illumination of the welding surface. When acquiring images, the image segmentation algorithm is used to remove background interference and only the image of the welding surface is retained.

[0124] The image analysis component performs steps S02 to S05 in any of the steel sheet defect detection methods in the first embodiment, that is, compares and analyzes image A and image B to obtain the difference K; determines whether K meets the preset standard; if so, continues to execute the loop until the entire steel sheet is covered; if not, it is determined that there is a defect. Then the large model is run to identify the defect type and classify the defect.

[0125] The steel sheet defect detection device provided in this embodiment can significantly improve the efficiency and accuracy of steel sheet defect detection, and is suitable for automated detection in actual production environments.

[0126] Embodiment 3

[0127] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steel sheet defect detection method of embodiment one is implemented. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a high-density digital video disc (DVD)), or a semiconductor medium (for example, a solid-state hard disk), etc. The computer-readable storage medium includes instructions that instruct the computing device to execute any one of the steel sheet defect detection methods provided in embodiment one or embodiment two.

[0128] Embodiment 4

[0129] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, that is, a computer program stored in the memory;

[0130] The processor executes the computer program to implement the steel sheet defect detection method in the first embodiment.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, other working processes of the method described above can refer to the corresponding processes in the aforementioned embodiments and will not be described in detail here.

[0132] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the method of each embodiment of the present application.

[0133] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0134] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integration. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state hard disk (SSD)), etc.

[0135] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting defects in a steel sheet, characterized in that: include: S1: Acquire the image of the steel sheet to be inspected to obtain the image to be inspected; S2: Input the image to be inspected into a defect detection model to identify the defect type and obtain the corresponding defect type. The training samples of the defect detection model include annotated images of one or more of the following defect types: rounded corner convex points, rounded corner concave points, rounded corner cracks, rounded corner dimensions that do not meet the standards, and deformation.

2. A steel sheet defect detection method according to claim 1, characterized in that: Before S1, it also included: S01: acquiring images of area A and area B of the steel sheet to obtain image A and image B, where area A and area B are symmetrical about the width axis of the steel sheet; S02: Compare and analyze image A and image B to obtain the difference K between image A and image B; S03: Determine whether K meets the preset standard; S04: If it meets the requirements, S01 to S03 are executed cyclically, and the area A corresponding to each cycle does not overlap with the union of the previous area A, and the area B corresponding to each cycle does not overlap with the union of the previous area B, until the union of all areas A and B coincides with the welding surface, it is determined to be qualified, and the steel sheet is sent to the processing station to execute the next production process; S05: If not, it is determined that there is a defect and step S1 is executed.

3. A steel sheet defect detection method according to claim 2, characterized in that: S05 includes: S051: If not, it is determined to be defective; S052: stitching image A and image B according to the outline of the steel sheet, the order and direction of image acquisition, to obtain a stitched image C; S053: Acquire image D, where image D is an image corresponding to the area on the steel sheet that is not covered by area A and area B; S1 includes: splicing image C and image D to obtain an image to be inspected.

4. A steel sheet defect detection method according to claim 2, characterized in that: S1 includes: S11: Acquire an image of the steel sheet to be inspected to obtain an image to be inspected; S12: determining preliminary defect information of the image to be inspected according to the difference K, wherein the preliminary defect information includes a defect area and / or a defect edge; S13: preprocessing and marking the image to be inspected according to the preliminary defect information; The defect detection model is provided with an attention mechanism, and the preliminary defect information serves as input of the attention mechanism.

5. A steel sheet defect detection method according to claim 2, characterized in that: The acquisition directions of image A and image B are opposite or opposite.

6. A steel sheet defect detection method according to any one of claims 1 to 5, characterized in that: S02 includes: Image A and image B are compared and analyzed by using any one or more of pixel level comparison, structural similarity index, feature matching and histogram comparison to obtain a difference K between image A and image B.

7. A steel sheet defect detection method according to claim 6, characterized in that: When the structural similarity index is used, S02 includes: S21: Convert image A and image B into grayscale images to obtain grayscale image A and grayscale image B; S22: For the corresponding positions of the grayscale image A and the grayscale image B, the local SSIM value is calculated using the selected window sliding; S23: Calculate the average of all local SSIM values ​​to obtain the global SSIM value.

8. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the steel sheet defect detection method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the steel sheet defect detection method according to any one of claims 1 to 7 are implemented.

10. A steel sheet defect detection device, characterized in that: include: A positioning component, used to fix the steel sheet at the image acquisition station; An image acquisition component, used for acquiring images of steel sheets; Image analysis component, used to analyze and process images.

Images