Deep Learning-Based Aluminum Alloy Surface Defect Detection Method

Abstract

The present invention relates to the field of image processing technology. More specifically, the present invention relates to a method for detecting aluminum alloy surface defects based on deep learning. The method includes collecting aluminum alloy images of each type and manually marking the defects, obtaining the prominence degree of each type of defect in the aluminum alloy images of each type, obtaining the discrimination degree of each aluminum alloy image with each type of defect according to the prominence degree and the discrimination degree between the non-defect area features and the defect area in the aluminum alloy images of each type, obtaining the gamma value of each aluminum alloy image with each type of defect according to the discrimination degree to enhance the image, obtaining the enhanced aluminum alloy image, using the enhanced aluminum alloy image to train the target detection model, inputting the latest collected aluminum alloy image into the trained target detection model to identify each defect and its defect type. The present invention increases the number of defect samples, enabling the target detection model to learn richer defect features and improving the accuracy of subsequent defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing. More specifically, the present invention relates to a method for detecting aluminum alloy surface defects based on deep learning. Background Art

[0002] The production of aluminum alloy involves multiple processes such as casting, rolling, and heat treatment. Defects in any one of these processes may cause surface defects in aluminum alloy such as bubbles, peeling, and sand holes. Surface defects will affect the use of aluminum alloy. In order to prevent defective aluminum alloy from being put into use, it is necessary to detect the surface defects of aluminum alloy.

[0003] It is known that deep learning technology for object detection can be used to detect surface defects of aluminum alloy. For defect detection based on deep learning, a large dataset of aluminum alloy images containing defects needs to be input into a neural network to train an object detection model. However, the surface defect features of aluminum alloy are often not obvious and the defect forms are diverse, making it difficult to accurately identify defects in images. As a result, there are fewer defect samples for model training, which affects the training results of the object detection model. Therefore, it is proposed to enhance the defects in aluminum alloy images to complete the shortage of defect samples, so that the training of the object detection model has sufficient sample data.

[0004] Since aluminum alloy exists in various forms such as aluminum ingots and aluminum bars, the collected aluminum alloy images also correspondingly contain multiple categories. For different forms of aluminum alloy, the surface texture, luster, and defect features in the images are different. The traditional gamma transformation sets a fixed gamma value and cannot enhance according to the characteristics of different forms of aluminum alloy images. It may cause that after some forms of aluminum alloy images are gamma-transformed, although the defects are enhanced to a certain extent, other important features of the images may also be changed, or the enhancement effect is not ideal, thus affecting the training effect of the model. Summary of the Invention

[0005] In order to solve the problem that the shortage of defect samples affects the training effect of the object detection model, the present invention proposes a method for detecting aluminum alloy surface defects based on deep learning. The method includes the following steps:

[0006] Collect a number of aluminum alloy images; classify them according to the form of aluminum alloy to obtain each category of aluminum alloy images; manually mark and obtain the defects and defect types of each aluminum alloy image; count each type of defect in each category of aluminum alloy images, and obtain the contrast and texture change degree of each type of defect in each category of aluminum alloy images; according to the contrast and texture change degree, obtain the prominence of each type of defect in each category of aluminum alloy images.

[0007] For each type of aluminum alloy image with defects, it is denoted as each type of defective aluminum alloy image, and conversely, it is denoted as each type of non-defective aluminum alloy image; obtain the non-defective feature vectors of each type of defective aluminum alloy image; obtain the standard feature vectors of each type of non-defective aluminum alloy image; according to the prominence degree, non-defective feature vectors, and standard feature vectors, obtain the discrimination degree of each type of defective aluminum alloy image; according to the discrimination degree, obtain the gamma value of each type of defective aluminum alloy image;

[0008] Use the gamma transformation algorithm to enhance each type of defective aluminum alloy image according to the gamma value of each type of defective aluminum alloy image, obtaining all enhanced aluminum alloy images; train an object detection model based on all enhanced aluminum alloy images; input the latest collected aluminum alloy images into the trained object detection model to identify defects.

[0009] The innovation of the present invention lies in obtaining the prominence degree of each type of defect in each type of aluminum alloy image through the contrast and texture change degree of each type of defect in each type of aluminum alloy image; obtaining the discrimination degree of each type of defective aluminum alloy image according to the prominence degree of each type of defect in each type of aluminum alloy image and the discrimination degree between the non-defective area features and the defective area in each type of aluminum alloy image, and adaptively enhancing the image according to the gamma value of each type of defective aluminum alloy image, which can enhance aluminum alloy images in different forms and improve the enhancement effect; training a neural network through the dataset after image enhancement to detect defects on the aluminum alloy surface, enabling the neural network to learn richer defect features and making the defect detection accuracy based on deep learning higher.

[0010] Preferably, the obtaining of the contrast and texture change degree of each type of defect in each type of aluminum alloy image includes:

[0011] According to the RMS contrast algorithm, obtain the contrast of each type of defect of the i-th type of aluminum alloy image; obtain the information entropy of the local binary pattern values of all pixel points in each type of defect of the i-th type of aluminum alloy image, which is denoted as the texture change degree of each type of defect of the i-th type of aluminum alloy image.

[0012] Preferably, the obtaining of the prominence degree of each type of defect in each type of aluminum alloy image includes:

[0013] Obtain the ratio of the number of pixel points of any type of defect of the i-th type of aluminum alloy image to the number of pixel points of the aluminum alloy image where the j-th type of defect is located, which is denoted as the area ratio of the j-th type of defect of the i-th type of aluminum alloy image;

[0014] ;

[0015] In the formula, represents the prominence of the j-th type of defect in the i-th type of aluminum alloy image; represents the average contrast of all the j-th type of defects in the i-th type of aluminum alloy image; represents the degree of texture change of all the j-th type of defects in the i-th type of aluminum alloy image; represents the average value of the area ratio of all the j-th type of defects in the i-th type of aluminum alloy image; norm() represents the normalization function.

[0016] It is convenient to adaptively obtain the gamma value of the image according to the prominence subsequently.

[0017] Preferably, obtaining the non-defect feature vector of each aluminum alloy image of each type of defect includes:

[0018] Count the types of local binary pattern values of the pixel points in the n-th aluminum alloy image of the i-th type of defect, use each type of local binary pattern value as the abscissa and the number of pixel points as the ordinate to construct an LBP histogram;

[0019] According to the LBP histogram, obtain the overall feature vector of the n-th aluminum alloy image of the i-th type of defect and the defect feature vector of the n-th aluminum alloy image of the i-th type of defect;

[0020] Subtract the defect feature vector of the n-th aluminum alloy image of the i-th type of defect from the overall feature vector of the n-th aluminum alloy image of the i-th type of defect to obtain the non-defect feature vector of the n-th aluminum alloy image of the i-th type of defect.

[0021] Preferably, obtaining the standard feature vector of each aluminum alloy image without defects includes:

[0022] Plot the local binary pattern values of the pixel points in any one aluminum alloy image of the i-th type without defects into the LBP histogram to obtain the standard LBP histogram of the aluminum alloy image of the i-th type without defects, and arrange the column heights of the standard LBP histogram in ascending order of the abscissa as the standard feature vector of the aluminum alloy image of the i-th type without defects.

[0023] Preferably, obtaining the overall feature vector of the n-th aluminum alloy image of the i-th type of defect and the defect feature vector of the n-th aluminum alloy image of the i-th type of defect includes:

[0024] Plot the local binary pattern values of all pixel points in the nth aluminum alloy image with the ith type of defect into the LBP histogram to obtain the overall LBP histogram. Arrange the bar heights of the overall LBP histogram in ascending order of the abscissa as the overall feature vector of the nth aluminum alloy image with the ith type of defect. Plot the local binary patterns of the pixel points in all defects of the nth aluminum alloy image with the ith type of defect into the LBP histogram to obtain the defect LBP histogram. Arrange the bar heights of the defect LBP histogram in ascending order of the abscissa as the defect feature vector of the nth aluminum alloy image with the ith type of defect.

[0025] Preferably, obtaining the discrimination degree of each aluminum alloy image with each type of defect includes:

[0026] ;

[0027] In the formula, represents the discrimination degree of the nth aluminum alloy image with the ith type of defect; represents the average prominence degree of all types of defects in the ith type of aluminum alloy image; represents the non-defect feature vector of the nth aluminum alloy image with the ith type of defect; represents the overall feature vector of the nth aluminum alloy image with the ith type of defect; represents the standard feature vector of the ith type of defect-free aluminum alloy image; represents the cosine similarity function; represents the sigmoid function.

[0028] The greater the discrimination degree, the less the aluminum alloy image needs to be enhanced, and the closer the gamma value is to 1.

[0029] Preferably, obtaining the gamma value of each aluminum alloy image with each type of defect includes:

[0030] Obtain the gamma value of the nth aluminum alloy image with the ith type of defect:

[0031] ;

[0032] In the formula, represents the gamma value of the nth aluminum alloy image with the ith type of defect; represents the discrimination degree of the nth aluminum alloy image with the ith type of defect; represents the average value of the discrimination degrees of all the ith type of aluminum alloy images with defects; represents the number of the ith type of aluminum alloy images; represents the number of the ith type of aluminum alloy images with defects; exp() represents the exponential function with the natural constant as the base; represents the average gray value of all pixel points in the nth aluminum alloy image with the ith type of defect.

[0033] The adaptive gamma value improves the accuracy of aluminum alloy image enhancement in different forms.

[0034] Preferably, training the object detection model according to all the enhanced aluminum alloy images includes:

[0035] Using the neural network YOLOv3, obtaining all the enhanced aluminum alloy images, manually marking the positions and types of defect areas in each enhanced aluminum alloy image with bounding boxes, and recording this marking result as the label of each enhanced aluminum alloy image; constituting a data set with all the enhanced aluminum alloy images and their corresponding labels; training this neural network with this data set, and using the mean square error loss function during the training process.

[0036] Enabling the neural network to learn richer defect features.

[0037] Preferably, the manually marking and obtaining the defects and defect types of each aluminum alloy image includes:

[0038] The annotator uses a special annotation tool to manually draw each target box in each aluminum alloy image to frame each defect in each aluminum alloy image, and mark the type of each defect in each aluminum alloy image; wherein, the image block corresponding to each manually drawn target box in each aluminum alloy image is each defect of each aluminum alloy image.

[0039] The present invention has the following beneficial effects: The purpose of the present invention is to obtain the prominence of each type of defect in each type of aluminum alloy image through the contrast and texture change degree of each type of defect in each type of aluminum alloy image; obtain the discrimination degree of each aluminum alloy image with each type of defect according to the prominence of each type of defect in each type of aluminum alloy image and the discrimination degree between the non-defect area features and the defect area in each type of aluminum alloy image, and adaptively adjust the gamma value of each aluminum alloy image with each type of defect to enhance the image according to the discrimination degree, so that the images with good discrimination degree are not over-enhanced, and the images with poor discrimination degree are enhanced, meeting the enhancement of aluminum alloy images in different forms and also improving the enhancement effect; training the neural network through the data set after image enhancement to detect the defects on the aluminum alloy surface, enabling the neural network to learn richer defect features. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present invention will become readily understood. In the drawings, several embodiments of the present invention are shown in an exemplary rather than restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0041] Figure 1 It is a flowchart of the steps of the aluminum alloy surface defect detection method based on deep learning in an embodiment of the present invention. Specific embodiments

[0042] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0043] Next, the specific embodiments of the present invention will be described in detail in conjunction with the accompanying drawings.

[0044] Please refer to Figure 1 , which shows a flowchart of the steps of the aluminum alloy surface defect detection method based on deep learning provided by an embodiment of the present invention. The method includes the following steps:

[0045] S001. After collecting a number of aluminum alloy images and classifying them, each type of aluminum alloy image is obtained, and the defect types in each type of aluminum alloy image are manually marked.

[0046] In an embodiment of the present invention, a number of aluminum alloy images are collected from a database. The annotator uses a special annotation tool to manually draw each target box in each aluminum alloy image to frame each defect in each aluminum alloy image, and mark the type (scratch, depression) of each defect in each aluminum alloy image; the image block corresponding to each manually drawn target box in each aluminum alloy image is each defect of each aluminum alloy image;

[0047] The collected number of aluminum alloy images are classified to obtain each type of aluminum alloy image. For example, the aluminum alloy images containing aluminum rods are used as one type of aluminum alloy image, and the aluminum alloy images containing aluminum ingots are used as one type of aluminum alloy image, etc.

[0048] S002. According to the contrast and texture change degree of each type of defect in each type of aluminum alloy image, obtain the prominence degree of each type of defect in each type of aluminum alloy image.

[0049] It should be noted that for defect detection based on deep learning, first, a large number of data sets of aluminum alloy images containing defects need to be input into the neural network to train the target detection model. However, since the surface defect features of aluminum alloys are often not obvious and the defect forms are diverse, it is difficult to accurately identify the defects in the images, resulting in fewer defect samples available for model training in the data set. Therefore, the present invention proposes to first enhance the defects in the aluminum alloy images to complete the shortage of defect samples, so that there are enough sample data for the training of the target detection model;

[0050] When the known gamma transformation algorithm enhances aluminum alloy images by setting a fixed gamma value, it cannot be flexibly adjusted according to the characteristics of aluminum alloy images in different forms. This may lead to the situation that after some aluminum alloy images are gamma-transformed, although the defects are enhanced to a certain extent, other important features of the images may be changed at the same time, or the enhancement effect is not ideal enough, thus affecting the training effect of the model. Therefore, in the present invention, by analyzing the marked defect features in each type of aluminum alloy image, the gamma value of the aluminum alloy image is adaptively enhanced.

[0051] It should be further noted that since the defects are obtained by manually marking the target boxes, the target boxes of the defects will include the surface defects of the aluminum alloy and the defect-free areas around the defects. If the contrast of the defects is greater, the texture change is more drastic, and the proportion of the defect area is larger, it indicates that the prominence degree of the defects is greater. Therefore, according to this feature, the prominence degree of each type of defect in each type of aluminum alloy image is obtained.

[0052] In the embodiment of the present invention, according to the RMS contrast algorithm, the contrast of each j-th type of defect in the i-th type of aluminum alloy image is obtained;

[0053] The information entropy of the local binary pattern values of all pixel points in each j-th type of defect in the i-th type of aluminum alloy image is obtained, and it is denoted as the texture change degree of each j-th type of defect in the i-th type of aluminum alloy image;

[0054] The ratio of the number of pixel points of any j-th type of defect in the i-th type of aluminum alloy image to the number of pixel points of the aluminum alloy image where the j-th type of defect is located is obtained, and it is denoted as the area ratio of the j-th type of defect in the i-th type of aluminum alloy image;

[0055] The prominence degree of each type of defect in each type of aluminum alloy image is obtained:

[0056] ;

[0057] In the formula, represents the prominence degree of the j-th type of defect in the i-th type of aluminum alloy image; represents the average contrast of all j-th type of defects in the i-th type of aluminum alloy image; represents the texture change degree of all j-th type of defects in the i-th type of aluminum alloy image; represents the average value of the area ratios of all j-th type of defects in the i-th type of aluminum alloy image; norm() represents the normalization function;

[0058] Since the defects are obtained by manually marking the target boxes, the target boxes of the defects will include the surface defects of the aluminum alloy and the defect-free areas around the defects. Therefore The larger it is, the greater the contrast between the aluminum alloy surface defects and the defect-free area around the defects within the target box of the j-th type of defect, and the more prominent the j-th type of defect is; The larger the value of is, the more drastic the texture change of the pixel points in the j-th type of defect in the i-th type of aluminum alloy image. At this time, the characteristics of the j-th type of defect are more obvious, and the more prominent the j-th type of defect is; The larger the value of is, the larger the area occupied by the j-th type of defect in the i-th type of aluminum alloy image, indicating that this type of defect is more prominent, and the more prominent the degree of this type of defect is.

[0059] S003. Obtain aluminum alloy images of each type of defect. According to the prominence of each type of defect in each type of aluminum alloy image and the degree of distinction between the non-defect area features and the defect area in each type of aluminum alloy image, obtain the discrimination degree of each aluminum alloy image of each type of defect. According to the discrimination degree of each aluminum alloy image of each type of defect, obtain the gamma value of each aluminum alloy image of each type of defect.

[0060] It should be noted that in the present invention, only the aluminum alloy images with defects need to be enhanced subsequently. Therefore, according to whether there are defects in each type of aluminum alloy image, each type of defective aluminum alloy image and each type of defect-free aluminum alloy image are obtained.

[0061] In the embodiments of the present invention, the images with defects in each type of aluminum alloy image are denoted as each type of defective aluminum alloy image; the images without defects in each type of aluminum alloy image are denoted as each type of defect-free aluminum alloy image.

[0062] It should be noted that due to the texture, reflection and other characteristics of the non-defect area on the aluminum alloy surface, sometimes they are extremely similar to the defect area. These two situations are very likely to cause problems of missed detection (inability to identify some real defects) or false detection (misjudging the normal area as a defect) in the target detection technology;

[0063] Therefore, the present invention considers whether the non-defect area features in the defective aluminum alloy image are easy to distinguish from the defect area. If they are easy to distinguish, it means that the distinction degree between the defect area and the non-defect area in the defective aluminum alloy image is greater, and the defects in the defective aluminum alloy image are more easily learned by the target detection model; and if the defective aluminum alloy image and the defect-free aluminum alloy image are more easily distinguished, it means that the defects in the defective aluminum alloy image have a greater impact on it. Therefore, it means that the distinction degree between the defect area and the non-defect area in the defective aluminum alloy image is greater, and the defects in the defective aluminum alloy image are more easily learned by the target detection model.

[0064] In the embodiments of the present invention, the types of local binary pattern values of the pixel points in the n-th defective aluminum alloy image of the i-th type are counted. Taking each type of local binary pattern value as the abscissa and the number of pixel points as the ordinate, an LBP histogram is constructed;

[0065] Plot the local binary pattern values of all pixel points in the n-th aluminum alloy image with the i-th type of defect into an LBP histogram to obtain the overall LBP histogram of the n-th aluminum alloy image with the i-th type of defect. Arrange the bar heights of the overall LBP histogram in ascending order of the abscissa as the overall feature vector of the n-th aluminum alloy image with the i-th type of defect;

[0066] Plot the local binary patterns of the pixel points in all defects of the n-th aluminum alloy image with the i-th type of defect into an LBP histogram to obtain the defect LBP histogram of the n-th aluminum alloy image with the i-th type of defect. Arrange the bar heights of the defect LBP histogram in ascending order of the abscissa as the defect feature vector of the n-th aluminum alloy image with the i-th type of defect;

[0067] Subtract the defect feature vector of the n-th aluminum alloy image with the i-th type of defect from the overall feature vector of the n-th aluminum alloy image with the i-th type of defect to obtain the non-defect feature vector of the n-th aluminum alloy image with the i-th type of defect;

[0068] Plot the local binary pattern values of the pixel points in any aluminum alloy image with the i-th type of no defect into an LBP histogram to obtain the standard LBP histogram of the aluminum alloy image with the i-th type of no defect. Arrange the bar heights of the standard LBP histogram in ascending order of the abscissa as the standard feature vector of the aluminum alloy image with the i-th type of no defect;

[0069] Obtain the discrimination degree of each aluminum alloy image with the i-th type of defect:

[0070] ;

[0071] wherein, represents the discrimination degree of the n-th aluminum alloy image with the i-th type of defect; represents the average prominence degree of all types of defects in the aluminum alloy image of the i-th type; represents the non-defect feature vector of the n-th aluminum alloy image with the i-th type of defect; represents the overall feature vector of the n-th aluminum alloy image with the i-th type of defect; represents the standard feature vector of the aluminum alloy image with the i-th type of no defect; represents the cosine similarity function; represents the sigmoid function; is used to perform negative correlation on ;

[0072] represents the similarity degree between the non-defective area and the whole image in the nth aluminum alloy image with the ith type of defect. The larger this value is, the more similar the non-defective area is to the whole image, which means the characteristics of the defective part have less influence on the characteristics of the whole image. Then, it is more difficult to distinguish the defect in the nth aluminum alloy image with the ith type of defect from the non-defective area, and the distinguishability of the nth aluminum alloy image with the ith type of defect is smaller. The smaller this value is, the greater the difference between the non-defective area and the whole image, which means the characteristics of the defective part have a greater influence on the characteristics of the whole image. Then, it is easier to distinguish the defect in the image from the non-defective area, and the distinguishability of the nth aluminum alloy image with the ith type of defect is larger.

[0073] represents the similarity degree between the nth aluminum alloy image with the ith type of defect and the aluminum alloy image with no defect of the ith type. The larger this value is, the smaller the influence of the defect in the nth aluminum alloy image with the ith type of defect on the image characteristics. At this time, it is more difficult to distinguish the image with a defect from the image without a defect, indicating that the distinguishability of the nth aluminum alloy image with the ith type of defect is smaller. The smaller this value is, the greater the influence of the defect in the nth aluminum alloy image with the ith type of defect on the image characteristics. Then, it is easier to distinguish the image with a defect from the image without a defect, and at this time, the distinguishability of the nth aluminum alloy image with the ith type of defect is larger.

[0074] The larger the value is, the more prominent the defect in the aluminum alloy image with the ith type of defect is, and the easier it is to distinguish the defect in the nth aluminum alloy image with the ith type of defect. Therefore, the distinguishability of the nth aluminum alloy image with the ith type of defect is larger.

[0075] It should be noted that when the gray value of any aluminum alloy image with the ith type of defect is larger, the gray value of this aluminum alloy image with the ith type of defect needs to be lowered. At this time, the gamma value of this aluminum alloy image with the ith type of defect should be greater than 1. Also, since the distinguishability of this aluminum alloy image with the ith type of defect is larger and the number of defect samples is more, it means that the defect in this aluminum alloy image with the ith type of defect is easier to distinguish from the defect-free image and the defect samples in the aluminum alloy image of the ith type are more abundant. Therefore, there is no need to over-enhance this aluminum alloy image with the ith type of defect, and the gamma value of this aluminum alloy image with the ith type of defect should be greater than 1 and close to 1. If the distinguishability of this aluminum alloy image with the ith type of defect is smaller and the number of defect samples is less, at this time, this aluminum alloy image with the ith type of defect needs to be enhanced, and the gamma value of this aluminum alloy image with the ith type of defect should be greater than 1 and far from 1.

[0076] If the gray value of any aluminum alloy image with the i-th type of defect is smaller, when performing gamma transformation enhancement on the aluminum alloy image with the i-th type of defect, it is necessary to increase the gray value of the aluminum alloy image with the i-th type of defect. At this time, the gamma value of the aluminum alloy image with the i-th type of defect should be less than 1. Also, since the greater the discrimination degree and the more defect samples in the aluminum alloy image with the i-th type of defect, it indicates that the defects in the aluminum alloy image with the i-th type of defect are more likely to be distinguished from the defect-free image and the defect samples in the aluminum alloy image of the i-th type are more abundant. Therefore, there is no need to over-enhance the aluminum alloy image with the i-th type of defect, and the gamma value of the aluminum alloy image with the i-th type of defect should be less than 1 and close to 1; if the discrimination degree of the aluminum alloy image with the i-th type of defect is smaller and the number of defect samples is less, at this time, it is necessary to enhance the aluminum alloy image with the i-th type of defect, and the gamma value of the aluminum alloy image with the i-th type of defect should be less than 1 and far from 1.

[0077] In the embodiment of the present invention, the gamma value of the n-th aluminum alloy image with the i-th type of defect is obtained:

[0078] ;

[0079] In the formula, represents the gamma value of the n-th aluminum alloy image with the i-th type of defect; represents the discrimination degree of the n-th aluminum alloy image with the i-th type of defect; represents the average value of the discrimination degrees of all aluminum alloy images with the i-th type of defect; represents the number of aluminum alloy images of the i-th type; represents the number of aluminum alloy images with the i-th type of defect; exp() represents the exponential function with the natural constant as the base; represents the average gray value of all pixel points in the n-th aluminum alloy image with the i-th type of defect.

[0080] The larger it is, it indicates that the gamma value of the n-th aluminum alloy image with the i-th type of defect should be greater than 1. At this time the value of and the value of

[0081] The smaller it is, it indicates that the gamma value of the n-th aluminum alloy image with the i-th type of defect should be less than 1. At this time the value of When the value is larger, the gamma value of the n-th aluminum alloy image with the i-th type of defect should be less than 1 and closer to 1; conversely, the gamma value of the n-th aluminum alloy image with the i-th type of defect should be less than 1 and farther from 1.

[0082] S004. Enhance the image according to the gamma value of each aluminum alloy image with each type of defect to obtain the enhanced aluminum alloy image. Use the enhanced aluminum alloy image to train the neural network, and input the latest collected aluminum alloy image into the trained neural network to identify the defects.

[0083] In the embodiment of the present invention, according to the gamma value of the n-th aluminum alloy image with the i-th type of defect, perform gamma transformation on the n-th aluminum alloy image with the i-th type of defect to obtain an enhanced aluminum alloy image. Similarly, obtain all the enhanced aluminum alloy images;

[0084] The neural network used in this embodiment is YOLOv3, and the method for training this neural network is:

[0085] Obtain all the enhanced aluminum alloy images, and manually mark the position of the defect area and the type of the defect area in each enhanced aluminum alloy image using a bounding box, and record this marking result as the label of each enhanced aluminum alloy image; constitute a data set with all the enhanced aluminum alloy images and their corresponding labels; use this data set to train this neural network, and the loss function used in the training process is the mean square error loss function; the specific training process is well-known content of the neural network, and the specific training process will not be elaborated in this embodiment.

[0086] Input the latest collected aluminum alloy image into the trained neural network to obtain the defect area and its defect type in the latest aluminum alloy image, such as scratches and dents.

[0087] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for detecting surface defects of aluminum alloy based on deep learning, characterized in that, Including: Collecting a number of aluminum alloy images; Classifying them according to the morphology of the aluminum alloy to obtain each type of aluminum alloy image; Manually marking and obtaining the defects and defect types of each type of aluminum alloy image; counting each type of defect in each type of aluminum alloy image, and obtaining the contrast and texture change degree of each type of defect of each type of aluminum alloy image; obtaining the prominence degree of each type of defect of each type of aluminum alloy image according to the contrast, texture change degree, and area ratio of each type of defect of each type of aluminum alloy image; Denoting the images with defects in each type of aluminum alloy image as each type of defective aluminum alloy image, and vice versa as each type of defect-free aluminum alloy image; obtaining the non-defect feature vector of each type of defective aluminum alloy image; Obtaining the standard feature vector of each type of defect-free aluminum alloy image; Obtain the discrimination degree of each aluminum alloy image with each type of defect ; where represents the discrimination degree of the nth aluminum alloy image with the ith type of defect; represents the average prominence degree of all types of defects in the ith type of aluminum alloy image; represents the non-defect feature vector of the nth aluminum alloy image with the ith type of defect; represents the overall feature vector of the nth aluminum alloy image with the ith type of defect; represents the standard feature vector of the ith type of defect-free aluminum alloy image; represents the cosine similarity function; represents the sigmoid function; According to the discrimination degree, obtain the gamma value of each aluminum alloy image with each type of defect; Using the gamma transformation algorithm to enhance each type of defective aluminum alloy image according to the gamma value of each type of defective aluminum alloy image to obtain all enhanced aluminum alloy images; Training an object detection model based on all enhanced aluminum alloy images; inputting the latest collected aluminum alloy images into the trained object detection model to identify defects.

2. The aluminum alloy surface defect detection method based on deep learning according to claim 1, wherein The obtaining of the contrast and texture change degree of each type of defect of each type of aluminum alloy image includes: Obtaining the contrast of each jth type of defect of the ith type of aluminum alloy image according to the RMS contrast algorithm; obtaining the information entropy of the local binary pattern values of all pixel points in each jth type of defect of the ith type of aluminum alloy image, which is denoted as the texture change degree of each jth type of defect of the ith type of aluminum alloy image.

3. The aluminum alloy surface defect detection method based on deep learning according to claim 1, characterized in that, The obtaining of the prominence degree of each type of defect of each type of aluminum alloy image includes: Obtaining the ratio of the number of pixel points of any jth type of defect of the ith type of aluminum alloy image to the number of pixel points of the aluminum alloy image where the jth type of defect is located, which is denoted as the area ratio of the jth type of defect of the ith type of aluminum alloy image; ； Wherein, represents the prominence of the j-th type of defect in the i-th type of aluminum alloy image; represents the average contrast of all the j-th type of defects in the i-th type of aluminum alloy image; represents the degree of texture change of all the j-th type of defects in the i-th type of aluminum alloy image; represents the average of the area ratios of all the j-th type of defects in the i-th type of aluminum alloy image; norm() represents the normalization function.

4. The aluminum alloy surface defect detection method based on deep learning according to claim 1, wherein The obtaining of the non-defect feature vector of each type of defective aluminum alloy image includes: Counting the types of local binary pattern values of the pixel points in the nth ith type of defective aluminum alloy image, using each type of local binary pattern value as the abscissa and the number of pixel points as the ordinate to construct an LBP histogram; Obtaining the overall feature vector and the defect feature vector of the nth ith type of defective aluminum alloy image according to the LBP histogram; Subtracting the defect feature vector of the nth ith type of defective aluminum alloy image from the overall feature vector of the nth ith type of defective aluminum alloy image to obtain the non-defect feature vector of the nth ith type of defective aluminum alloy image.

5. The method for detecting surface defects of aluminum alloy based on deep learning according to claim 1 or 4, characterized in that, The obtaining of the standard feature vector of each type of defect-free aluminum alloy image includes: Plotting the local binary pattern values of the pixel points in any one ith type of defect-free aluminum alloy image into the LBP histogram to obtain the standard LBP histogram of the ith type of defect-free aluminum alloy image, and arranging the column heights of the standard LBP histogram in ascending order of the abscissa as the standard feature vector of the ith type of defect-free aluminum alloy image.

6. The method for detecting surface defects of aluminum alloy based on deep learning according to claim 4, wherein The obtaining of the overall feature vector and the defect feature vector of the nth ith type of defective aluminum alloy image includes: Plot the local binary pattern values of all pixel points in the nth aluminum alloy image with the i-th type of defect into the LBP histogram to obtain the overall LBP histogram. Arrange the bar heights of the overall LBP histogram in ascending order of the abscissa as the overall feature vector of the nth aluminum alloy image with the i-th type of defect. Plot the local binary patterns of the pixel points in all defects of the nth aluminum alloy image with the i-th type of defect into the LBP histogram to obtain the defect LBP histogram. Arrange the bar heights of the defect LBP histogram in ascending order of the abscissa as the defect feature vector of the nth aluminum alloy image with the i-th type of defect.

7. The aluminum alloy surface defect detection method based on deep learning according to claim 1, wherein The obtaining of the gamma value of each aluminum alloy image with each type of defect includes: Obtaining the gamma value of the nth aluminum alloy image with the i-th type of defect: ； In the formula, represents the gamma value of the nth aluminum alloy image with the ith type of defect; represents the discrimination degree of the nth aluminum alloy image with the ith type of defect; represents the mean value of the discrimination degrees of all aluminum alloy images with the ith type of defect; represents the number of aluminum alloy images of the ith type; represents the number of aluminum alloy images with the ith type of defect; exp() represents the exponential function with the natural constant as the base; represents the mean gray value of all pixel points in the nth aluminum alloy image with the ith type of defect.

8. The method for detecting surface defects of aluminum alloy based on deep learning according to claim 1, wherein The training of the object detection model according to all enhanced aluminum alloy images includes: Using the neural network YOLOv3, obtain all enhanced aluminum alloy images. Manually mark the position of the defect area and the type of the defect area in each enhanced aluminum alloy image using bounding boxes, and record this marking result as the label of each enhanced aluminum alloy image. Construct a data set from all enhanced aluminum alloy images and their corresponding labels. Use this data set to train the neural network, and the loss function used in the training process is the mean square error loss function.

9. The aluminum alloy surface defect detection method based on deep learning according to claim 1, characterized in that The manually marking and obtaining of the defects and defect types of each aluminum alloy image includes: The annotator uses a dedicated annotation tool to manually draw each target box in each aluminum alloy image to frame each defect in each aluminum alloy image, and mark the type of each defect in each aluminum alloy image. Among them, the image block corresponding to each manually drawn target box in each aluminum alloy image is each defect of each aluminum alloy image.

Images