Surface Defect Detection Method, Apparatus, Device, and Storage Medium

Abstract

The present application discloses a surface defect detection method, device, equipment and storage medium, relating to the technical field of image processing. The method includes: obtaining an image to be detected, and performing saliency detection on the image to be detected by using at least two preset detection algorithms to obtain corresponding saliency images; performing superpixel segmentation on the image to be detected by using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and extracting features of each superpixel to obtain the image features of each superpixel; based on each saliency image and the image features of each superpixel and by using a preset difference function, determining the difference value between the background and the saliency region of each saliency image; and determining a defect detection result based on the difference values of each saliency image. Thereby, the accuracy of surface defect detection is improved, and the problem that different images are difficult to accurately adapt to the saliency detection algorithm with the optimal performance is solved.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and particularly to a surface defect detection method, apparatus, device, and storage medium. Background Art

[0002] Salience detection technology is mainly used to find the image regions in an image that can arouse the interest of human visual perception, and is the basis for various tasks in computer vision, such as image cropping, image compression, image stitching, image classification, and target recognition.

[0003] However, due to the complexity of the perception mechanism of the human visual system, there are currently a variety of salience detection algorithms designed from different perspectives; in addition, the visual patterns contained in different images are complex and diverse, so for different images, different salience detection algorithms are often adapted. For example, for the defect detection task on the surface of an aircraft, due to the characteristics of the aircraft's titanium alloy fuselage, such as large curved surfaces, complex structures, and high manufacturing precision, the probability of surface defects is low, the image metal texture is complex, the defect area is small, and the background-defect contrast is low due to the dull metal luster. Therefore, it is impossible to accurately detect the surface defects of an aircraft using the existing salience detection technology. Summary of the Invention

[0004] The main purpose of the present application is to provide a surface defect detection method, apparatus, device, and storage medium to improve the accuracy of surface defect detection and solve the problem that it is difficult to accurately adapt different images to the salience detection algorithm with the best performance.

[0005] To achieve the above object, the present application provides a surface defect detection method, including:

[0006] Obtain an image to be detected, and perform salience detection on the image to be detected using at least two preset detection algorithms to obtain corresponding salience images;

[0007] Perform superpixel segmentation on the image to be detected using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and extract features of each superpixel to obtain the image features of each superpixel;

[0008] Based on each salience image and the image features of each superpixel and using a preset difference function, determine the difference value between the background and the salience region of each salience image;

[0009] Determine the defect detection result based on the difference values of each salience image.

[0010] Optionally, performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain corresponding saliency images, including: performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain corresponding initial saliency images; adaptively segmenting each of the initial saliency images to obtain each of the saliency images.

[0011] Optionally, determining the difference degree values between the background and the saliency regions of each of the saliency images based on the image features of each of the saliency images and each of the superpixels and using a preset difference degree function, including: determining the superpixels of the background region and the superpixels of the saliency region based on the pixel values of each of the saliency images and the pixel values of each of the superpixels; determining the difference degree values between the background and the saliency regions of each of the saliency images based on the image features of each of the superpixels of the background region and the image features of each of the superpixels of the saliency region.

[0012] Optionally, determining the superpixels of the background region and the superpixels of the saliency region based on the pixel values of each of the saliency images and the pixel values of each of the superpixels, including: for any saliency image, defining the pixel points with the pixel value of the first preset value in the saliency image as background pixel points, and defining the pixel points with the pixel value of the second preset value in the saliency image as saliency pixel points; for any superpixel, defining the superpixel as the superpixel of the background region when the background pixel points in the superpixel exceed the preset proportion, and defining the superpixel as the superpixel of the saliency region when the saliency pixel points in the superpixel exceed the preset proportion.

[0013] Optionally, the preset difference degree function is:

[0014]

[0015] where is the preset difference degree function, is the saliency image, I is the image to be detected, is the maximum mean difference calculation function, and are the feature sets of the superpixels of the background region and the superpixels of the saliency region respectively, and are the numbers of the superpixels of the background region and the superpixels of the saliency region respectively, is the radial basis kernel function, and are the image features of the i-th superpixel of the background region, and are the image features of the j-th superpixel of the saliency region.

[0016] Optionally, determining the defect detection result based on the difference degree values of the respective saliency images includes: selecting, from the difference degree values of the respective saliency images, the saliency image corresponding to the maximum difference degree value as the target saliency image; if the saliency region exists in the target saliency image, determining that the defect detection result corresponding to the image to be detected is that a defect exists; if the saliency region does not exist in the target saliency image, determining that the defect detection result corresponding to the image to be detected is that no defect exists.

[0017] Optionally, the image features of the superpixels at least include the mean gray percentage, image entropy, and Gabor features.

[0018] In addition, to achieve the above object, the present application further provides a surface defect detection device, including: a saliency detection module, configured to obtain an image to be detected and perform saliency detection on the image to be detected by using at least two preset detection algorithms to obtain a corresponding saliency image; a superpixel segmentation module, configured to perform superpixel segmentation on the image to be detected by using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and extract features of each of the superpixels to obtain the image features of each of the superpixels; a difference degree calculation module, configured to determine the difference degree values between the background and the saliency regions of the respective saliency images based on the respective saliency images and the image features of the respective superpixels by using a preset difference degree function; and a result determination module, configured to determine the defect detection result based on the difference degree values of the respective saliency images.

[0019] The present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the surface defect detection method described in any one of the above is implemented.

[0020] The present application further provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the surface defect detection method described in any one of the above is implemented.

[0021] The surface defect detection method of the present application performs saliency detection on the image to be detected by using a variety of preset detection algorithms, and obtains saliency images corresponding to each preset detection algorithm; then uses a preset superpixel segmentation algorithm to perform superpixel segmentation on the image to be detected, obtains a plurality of superpixels, and extracts features of each superpixel to obtain the image features of each superpixel; further determines the difference degree value between the background and the saliency region of each saliency image based on each saliency image and the image features of each superpixel and by using a preset difference degree function; finally determines the defect detection result based on the difference degree values of each saliency image. By integrating the detection results of multiple saliency detection algorithms with the results of the superpixel segmentation algorithm, it is possible to match the saliency detection algorithm with the optimal performance for the image to be detected, thereby improving the accuracy of surface defect detection. Description of the Drawings

[0022] Figure 1 is one of the flowcharts of the surface defect detection method according to an embodiment of the present application;

[0023] Figure 2(a) is the original image of the image to be detected in a specific example of the present application;

[0024] Figure 2(b) is the initial saliency image output by the BC algorithm model in a specific example of the present application;

[0025] Figure 2(c) is the initial saliency image output by the FT algorithm model in a specific example of the present application;

[0026] Figure 2(d) is the initial saliency image output by the GS algorithm model in a specific example of the present application;

[0027] Figure 2(e) is the initial saliency image output by the MCITF algorithm model in a specific example of the present application;

[0028] Figure 3(a) is the result image after adaptive segmentation of the initial saliency image corresponding to the BC algorithm in a specific example of the present application;

[0029] Figure 3(b) is the result image after adaptive segmentation of the initial saliency image corresponding to the FT algorithm in a specific example of the present application;

[0030] Figure 3(c) is the result image after adaptive segmentation of the initial saliency image corresponding to the GS algorithm in a specific example of the present application;

[0031] Figure 3(d) is the result image after adaptive segmentation of the initial saliency image corresponding to the MCITF algorithm in a specific example of the present application;

[0032] Figure 4 is a schematic diagram of the superpixel segmentation result in a specific example of the present application;

[0033] Figure 5 It is the second flow chart of the surface defect detection method according to an embodiment of the present application;

[0034] Figure 6 It is the third flow chart of the surface defect detection method according to an embodiment of the present application;

[0035] Figure 7 It is a schematic structural diagram of a surface defect detection device according to an embodiment of the present application;

[0036] Figure 8 It exemplifies a schematic physical structure diagram of an electronic device;

[0037] In the figure, 700 is a surface defect detection device; 710 is a saliency detection module; 720 is a superpixel segmentation module; 730 is a difference degree calculation module; 740 is a result determination module; 810 is a processor; 820 is a communication interface; 830 is a memory; 840 is a communication bus.

[0038] The realization, functional features and advantages of the purpose of the present application will be further described in conjunction with the embodiments with reference to the accompanying drawings. Specific embodiments

[0039] To make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0040] The saliency detection algorithm imitates the human visual system and detects the salient regions by analyzing the differences in different regions of the image. However, due to the complexity of the perception mechanism of the human visual system, existing methods are designed from different perspectives, and this theoretical design difference leads to certain differences in the detection results of different algorithms when detecting the same image. In addition, the visual patterns contained in different images are complex and diverse, so different images often adapt to different saliency detection algorithms.

[0041] Especially in the application scenario of aircraft fuselage surface defect detection, due to the characteristics of the aircraft titanium alloy fuselage such as large curved surface, complex structure, and high manufacturing precision, the probability of surface defects is low, the metal texture of the image is complex, the defect area is small, and the background-defect contrast is low due to the dim metal luster.

[0042] Currently, the existing saliency detection algorithms mainly detect surface defects by generating saliency maps. The existing saliency detection algorithms need to collect training samples in advance and provide ground truth values to train the classifier. At the same time, only a single saliency model is used to judge the presence or absence of defects, lacking multi-angle and multi-dimensional information. Therefore, only using the existing saliency detection algorithms to detect the surface defects of the aircraft body, the detection accuracy is low and the model training is complex.

[0043] Based on this, the embodiments of the present application provide a surface defect detection method, device, equipment and storage medium. By integrating multiple saliency detection algorithms and comprehensively considering multi-angle and multi-dimensional information, the accuracy of surface defect detection is improved. And the embodiments of the present application do not need to collect training samples to provide ground truth values, so they are more suitable for small sample scenarios, solving the problem that it is difficult to accurately adapt different images to the saliency detection algorithm with the best performance in the face of a variety of existing saliency detection algorithms.

[0044] Figure 1 is one of the flowcharts of the surface defect detection method in the embodiments of the present application. This surface defect detection method can be executed by the processor of any electronic device, such as Figure 1 shown, this surface defect detection method may include the following steps:

[0045] Step 110: Obtain the image to be detected, and perform saliency detection on the image to be detected by using at least two preset detection algorithms to obtain corresponding saliency images.

[0046] Step 120: Use a preset superpixel segmentation algorithm to perform superpixel segmentation on the image to be detected to obtain multiple superpixels, and extract features from each superpixel to obtain the image features of each superpixel.

[0047] Step 130: Based on each saliency image and the image features of each superpixel, and using a preset difference function, determine the difference value between the background and the saliency region of each saliency image.

[0048] Step 140: Determine the defect detection result based on the difference values of each saliency image.

[0049] First of all, it should be noted that the embodiments of the present application can be applied to any metal surface defect detection scenario, especially in the scenario of detecting the surface defects of titanium alloy metal on the aircraft body. The defective rate in the processing process of titanium alloy metal is very low. If traditional deep learning-based feature extraction and fusion methods are used, it is difficult to collect sufficient training data. Therefore, the scenario of detecting the surface defects of titanium alloy metal on the aircraft body is a typical small sample scenario, and the ground truth values (i.e., the saliency regions containing surface defects) often need to be manually marked, and the cost is relatively high.

[0050] In step 110, the image to be detected can be collected by a camera. After obtaining the image to be detected, the image to be detected can be input into multiple preset detection algorithms simultaneously. Each preset detection algorithm performs saliency detection on the image to be detected to obtain a saliency image corresponding to each preset detection algorithm.

[0051] In this embodiment, multiple preset detection algorithms may include (but are not limited to): MCITF saliency detection algorithm (Saliency detection for strip steel surface defects using multiple constraints and improved texture features), BC saliency detection algorithm (Saliency Optimization from Robust Background Detection), FT saliency detection algorithm (Frequency-Tuned Salient Region Detection), and GS saliency detection algorithm (Geodesic Saliency Using Background Prior).

[0052] Taking the above four saliency detection algorithms as preset detection algorithms as an example, the surface defect detection method of the embodiment of the present application will be introduced in detail below.

[0053] In some embodiments, step 110 of using at least two preset detection algorithms to perform saliency detection on the image to be detected to obtain corresponding saliency images may include: using at least two preset detection algorithms to perform saliency detection on the image to be detected to obtain corresponding initial saliency images; and performing adaptive segmentation on each initial saliency image to obtain each saliency image.

[0054] Specifically, after obtaining the image to be detected, the image to be detected can be input into the MCITF algorithm model, BC algorithm model, FT algorithm model, and GS algorithm model simultaneously. The algorithm models of each preset detection algorithm perform saliency detection on the image to be detected respectively and output corresponding initial saliency images. In this embodiment, the detection results output by the above four preset detection algorithms can be respectively denoted as: 、 、 、 。

[0055] Figure 2(a) is the original image of the image to be detected in a specific example of this application; Figure 2(b) is the initial saliency image output by the BC algorithm model in a specific example of this application; Figure 2(c) is the initial saliency image output by the FT algorithm model in a specific example of this application; Figure 2(d) is the initial saliency image output by the GS algorithm model in a specific example of this application; Figure 2(e) is the initial saliency image output by the MCITF algorithm model in a specific example of this application. It can be seen from Figure 2(a) - Figure 2(e) that through the above four preset detection models, the saliency regions and background regions in the image to be detected are distinguished. However, when different saliency detection algorithms perform saliency detection on the same image to be detected, the images of the saliency detection results output are all different.

[0056] It should be noted that the structures and detection processes of the above four saliency detection algorithms can refer to the structures and detection processes of existing saliency detection algorithms, which will not be elaborated here.

[0057] After obtaining the initial saliency images output by each preset detection algorithm, the initial saliency images can be adaptively segmented to obtain each saliency image. Specifically, the maximum inter-class variance method can be used to perform binary processing on the above calculation results respectively, and all the processing results are denoted as , .

[0058] It should be noted that the maximum inter-class variance method finds a suitable threshold by traversing the gray level histogram of the initial saliency image; then, by comparing the pixel value of each pixel point in the initial saliency image with the threshold, the pixel points in the initial saliency image are divided into pixel points with the first preset value and pixel points with the second preset value.

[0059] As an example, first process the input initial saliency image to obtain its grayscale histogram. This histogram describes the occurrence frequency of different gray levels in the image. Further traverse each possible gray level as a candidate threshold. For each candidate threshold, divide all the pixels in the image into two categories. The first category (foreground): all pixels with pixel values greater than or equal to the threshold; the second category (background): all pixels with pixel values less than the threshold. Next, calculate the within-class variance and between-class variance: for each category, calculate their means and their respective within-class variances (i.e., the degree of dispersion of the pixel value distribution within the class). Then calculate the between-class variance, which is the weighted sum of the square of the difference between the means of the two categories and their respective proportions of the occupied image. Select the threshold that maximizes the between-class variance as the final segmentation threshold. The purpose of this step is to ensure the maximum difference between the two parts (foreground and background) after segmentation. Use the found optimal threshold to binarize the original image. For each pixel point in the image, compare its pixel value with the optimal threshold: if the pixel value is less than the threshold, mark the pixel point as "background" and assign it the first preset value (usually black, i.e., 0); if the pixel value is greater than or equal to the threshold, mark the pixel point as "foreground" and assign it the second preset value (usually white, i.e., 255).

[0060] Figure 3(a) is the result image after adaptive segmentation of the initial saliency image corresponding to the BC algorithm in a specific example of this application; Figure 3(b) is the result image after adaptive segmentation of the initial saliency image corresponding to the FT algorithm in a specific example of this application; Figure 3(c) is the result image after adaptive segmentation of the initial saliency image corresponding to the GS algorithm in a specific example of this application; Figure 3(d) is the result image after adaptive segmentation of the initial saliency image corresponding to the MCITF algorithm in a specific example of this application. It can be seen from Figure 3(a) - Figure 3(d) that after the initial saliency image undergoes adaptive segmentation, all pixel points are divided into two types according to the pixel values, and the saliency region and the background region can be clearly seen.

[0061] In step 120, after obtaining the image to be detected, the preset superpixel segmentation algorithm can be used to perform superpixel segmentation on the image to be detected. In this embodiment, the simple linear iterative clustering algorithm can be used to perform superpixel segmentation on the image to be detected, so as to segment the image to be detected into several superpixels according to the local pixel similarity. Specifically, superpixel segmentation is based on the similarity of features such as color, brightness, and texture between the pixels of the image to be detected, and the image is subdivided into multiple small regions, and these small regions are superpixels.

[0062] Figure 4 is a schematic diagram of the superpixel segmentation result of a specific example of this application. As Figure 4As shown, through superpixel segmentation, the image to be detected is segmented into several small regions (i.e., superpixels), and the pixel features of each superpixel are similar. From Figure 4 the segmentation results, it can also be obtained which superpixels are significant regions and which superpixels are background regions. It should be noted that the preset superpixel segmentation algorithm can adopt existing superpixel segmentation algorithms, such as the K-means algorithm, the Mean Shift algorithm, etc. The specific process of superpixel segmentation can also refer to the existing superpixel segmentation process, which will not be elaborated here.

[0063] Furthermore, after superpixel segmentation to obtain several superpixels, feature extraction can be performed on each superpixel to obtain the image features of each superpixel. In some embodiments, the image features of the superpixels at least include the mean gray percentage , image entropy and Gabor features.

[0064] Among them, the mean gray percentage refers to: for a given superpixel region, calculate the average value of the gray values of all pixels. This can be used to characterize the overall brightness level of the superpixel. Secondly, image entropy can measure the uncertainty or complexity of information in the image. For a superpixel, the entropy can be calculated based on the gray histogram of its internal pixels. The higher the entropy, the greater the gray change within the superpixel and the higher the information content. Finally, Gabor features are features extracted based on Gabor filters; these filters can capture information in different directions and frequencies in the image. For a superpixel, texture information can be extracted by applying the Gabor filter to the region and calculating the response intensity. Commonly used Gabor features include the response intensity in terms of direction and scale. The specific process of image feature extraction can refer to the existing image feature extraction process, which will not be elaborated here.

[0065] In addition, it should also be noted that the order of step 110 and step 120 can be swapped. In short, it is also possible to first perform superpixel segmentation on the image to be detected using the preset superpixel segmentation algorithm, and then perform saliency detection on the image to be detected using multiple preset detection algorithms. This embodiment only provides an example process of a surface defect detection method, and should not limit the protection scope of the present application.

[0066] Furthermore, in step 130, after obtaining the saliency images corresponding to each preset detection algorithm and the image features of each superpixel, a preset difference function established in advance can be used to calculate the weights of the saliency images corresponding to each preset detection algorithm, so as to quantitatively evaluate the detection results of each preset detection algorithm, so as to select the most accurate detection result from all saliency detection results subsequently.

[0067] Figure 5 This is the second flowchart of the surface defect detection method according to the embodiments of the present application. As Figure 5 shown, in some embodiments, step 130 determines the difference degree values between the background and the significant regions of each significant image based on the image features of each significant image and each superpixel and using a preset difference degree function, which may include the following steps:

[0068] Step 510: Determine the superpixels of the background region and the superpixels of the significant region based on the pixel values of each significant image and the pixel values of each superpixel.

[0069] Step 520: Determine the difference degree values between the background and the significant regions of each significant image based on the image features of each superpixel in the background region and the image features of each superpixel in the significant region.

[0070] In this embodiment, since each significant image has been binarized, therefore, the background region and the significant region can be distinguished based on the pixel values of the significant image. The embodiments of the present application combine the binarization result of the significant image and the superpixel segmentation result to obtain the corresponding background region and significant region.

[0071] It should be noted that the background region and the significant region determined by using the binarization results of different significant images may also be different.

[0072] In some embodiments, determining the superpixels of the background region and the superpixels of the significant region based on the pixel values of each significant image and the pixel values of each superpixel may include: for any significant image, defining the pixel points with the pixel value of the first preset value in the significant image as background pixel points, and defining the pixel points with the pixel value of the second preset value in the significant image as significant pixel points; for any superpixel, in the case where the background pixel points in the superpixel exceed the preset proportion, defining the superpixel as a superpixel of the background region, and in the case where the significant pixel points in the superpixel exceed the preset proportion, defining the superpixel as a superpixel of the significant region.

[0073] It should be noted that the first preset value, the second preset value, and the preset proportion can be set according to actual needs. For example, the first preset value can be set to 0, the second preset value can be set to 1, and the preset proportion can be set to 50%. Here, the first preset value, the second preset value, and the preset proportion are not specifically limited.

[0074] The following takes the first preset value as 0, the second preset value as 1, and the preset proportion as 50% as an example to introduce this step in detail.

[0075] Taking a saliency image as an example, the pixel points with a pixel value of 0 in the binarized saliency image can be defined as background pixel points, and the pixel points with a pixel value of 1 in the binarized saliency image can be defined as saliency pixel points. Then, the definition results of each pixel point in the above saliency image are applied to the superpixels. For a certain superpixel, if the proportion of background pixel points in the superpixel exceeds 50%, then the superpixel can be defined as a background region superpixel; if the proportion of saliency pixel points in the superpixel exceeds 50%, then the superpixel can be defined as a saliency region superpixel. Thus, the superpixels and the saliency image can be combined to determine the background region superpixels and the saliency region superpixels.

[0076] In the same way, the background region superpixels and the saliency region superpixels corresponding to each saliency image can be determined. Further, after determining the background region superpixels and the saliency region superpixels corresponding to each saliency image, the difference degree value of the background-saliency region can be calculated using a preset difference degree function. It can be understood that the larger the difference degree value of the background-saliency region, the more obvious the difference between the saliency region and the surrounding background, and thus it is easier to be perceived or detected. At this time, it is considered that the segmentation effect of the saliency image is the best.

[0077] In some embodiments, the preset difference degree function can be:

[0078]

[0079] Among them, is the preset difference degree function, is the saliency image, I is the image to be detected, is the maximum mean difference calculation function, and are the feature sets of the background region superpixels and the saliency region superpixels respectively, and are the numbers of the background region superpixels and the saliency region superpixels respectively, is the radial basis kernel function, and are the image features of the i-th background region superpixel, and are the image features of the j-th saliency region superpixel.

[0080] After determining the background region superpixels and the saliency region superpixels corresponding to each saliency image and the image features of each superpixel, the above parameters can be substituted into the preset difference degree function, and the difference degree values of the background-saliency region corresponding to each preset detection algorithm can be obtained respectively.

[0081] After obtaining the difference values of the background - saliency regions corresponding to the preset detection algorithms in step 140, the defect detection result can be determined based on the difference values of the saliency images. Figure 6 is the third flowchart of the surface defect detection method according to the embodiment of the present application. As Figure 6 shown, in some embodiments, determining the defect detection result based on the difference values of the saliency images may include the following steps:

[0082] Step 610: Select the saliency image corresponding to the maximum difference value among the difference values of the saliency images as the target saliency image.

[0083] Step 620: If there is a saliency region in the target saliency image, determine that the defect detection result corresponding to the image to be detected is that there is a defect.

[0084] Step 630: If there is no saliency region in the target saliency image, determine that the defect detection result corresponding to the image to be detected is that there is no defect.

[0085] Specifically, after calculating the difference values of the saliency images, all the difference values can be compared, and the saliency image with the maximum difference value is used as the target saliency image. In this embodiment, step 610 can be implemented using the following formula:

[0086]

[0087] where is the difference value of the finally output target saliency image, , , , are the background - saliency region difference values of the saliency images corresponding to the MCITF algorithm, FT algorithm, BC algorithm, and GS algorithm, respectively.

[0088] Furthermore, after determining the target saliency image with the best segmentation effect, it can be determined whether there is a defect in the image to be detected by judging whether there is a saliency region in the target saliency image. Specifically, if there is a region with a second preset value in the target saliency image, it can be determined that there is a defect in the image to be detected, and the region with the second preset value is the defect; if there is no region with a second preset value in the target saliency image, it can be determined that there is no defect in the image to be detected.

[0089] Therefore, compared with traditional saliency surface defect detection algorithms, the embodiments of the present application can significantly improve the accuracy of surface defect detection by integrating multiple saliency detection algorithms and comprehensively considering multi-angle and multi-dimensional information. Secondly, the defect detection method of the embodiments of the present application has high scalability. If a new saliency detection algorithm appears, the new saliency detection algorithm can be directly added to this method. Finally, the defect detection method provided by the embodiments of the present application does not require collecting training samples or providing ground truth values, and is more suitable for small-sample scenarios similar to the aircraft body defect detection scenario.

[0090] Based on the above embodiments, the embodiments of the present application further provide a surface defect detection device. Figure 7 It is a schematic structural diagram of the surface defect detection device of the embodiments of the present application. As Figure 7 shown, the surface defect detection device 700 may include: a saliency detection module 710, a superpixel segmentation module 720, a difference calculation module 730, and a result determination module 740. The saliency detection module 710, the superpixel segmentation module 720, and the result determination module 740 are respectively connected to the difference calculation module 730.

[0091] Among them, the saliency detection module 710 is used to obtain the image to be detected, and perform saliency detection on the image to be detected by using at least two preset detection algorithms to obtain the corresponding saliency images; the superpixel segmentation module 720 is used to perform superpixel segmentation on the image to be detected by using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and extract features of each superpixel to obtain the image features of each superpixel; the difference calculation module 730 is used to determine the difference value between the background and the saliency region of each saliency image based on each saliency image and the image features of each superpixel and by using a preset difference function; the result determination module 740 is used to determine the defect detection result based on the difference values of each saliency image.

[0092] Therefore, the saliency detection module 710 performs saliency detection on the image to be detected by using multiple preset detection algorithms to obtain saliency images corresponding to the preset detection algorithms; the superpixel segmentation module 720 then performs superpixel segmentation on the image to be detected by using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and extracts features of each superpixel to obtain the image features of each superpixel; further, the difference calculation module 730 determines the difference value between the background and the saliency region of each saliency image based on each saliency image and the image features of each superpixel and by using a preset difference function; finally, the result determination module 740 determines the defect detection result based on the difference values of each saliency image. By integrating the detection results of multiple saliency detection algorithms with the results of the superpixel segmentation algorithm, the saliency detection algorithm with the optimal performance can be matched for the image to be detected, thereby improving the accuracy of surface defect detection.

[0093] In some embodiments, the saliency detection module 710 is specifically configured to: perform saliency detection on the image to be detected by using at least two preset detection algorithms to obtain corresponding initial saliency images; and perform adaptive segmentation on each initial saliency image to obtain each saliency image.

[0094] In some embodiments, the difference degree calculation module 730 is specifically configured to: determine background region superpixels and saliency region superpixels based on the pixel values of each saliency image and the pixel values of each superpixel; and determine the difference degree values between the background and the saliency regions of each saliency image based on the image features of each background region superpixel and the image features of each saliency region superpixel.

[0095] In some embodiments, the difference degree calculation module 730 is further specifically configured to: for any saliency image, define the pixel points with the pixel value of the first preset value in the saliency image as background pixel points, and define the pixel points with the pixel value of the second preset value in the saliency image as saliency pixel points; for any superpixel, if the background pixel points in the superpixel exceed a preset proportion, define the superpixel as a background region superpixel, and if the saliency pixel points in the superpixel exceed a preset proportion, define the superpixel as a saliency region superpixel.

[0096] In some embodiments, the preset difference degree function is:

[0097]

[0098] Wherein, is the preset difference degree function, is the saliency image, I is the image to be detected, is the maximum mean difference calculation function, and are the feature sets of the background region superpixels and the saliency region superpixels respectively, and are the numbers of the background region superpixels and the saliency region superpixels respectively, is the radial basis kernel function, and are the image features of the i-th background region superpixel, and are the image features of the j-th saliency region superpixel.

[0099] In some embodiments, the result determination module 740 is specifically configured to: select the saliency image corresponding to the maximum difference value from the difference values of the saliency images as the target saliency image; if there is a salient region in the target saliency image, determine that the defect detection result corresponding to the image to be detected is that there is a defect; if there is no salient region in the target saliency image, determine that the defect detection result corresponding to the image to be detected is that there is no defect.

[0100] In some embodiments, the image features of the superpixels at least include the mean gray percentage, image entropy, and Gabor features.

[0101] It should be noted that for the details not disclosed in the surface defect detection device of this embodiment, please refer to the details disclosed in the embodiments of the surface defect detection method in this specification, which will not be elaborated here.

[0102] Figure 8 An example of the physical structure diagram of an electronic device is shown as Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the surface defect detection method, which includes: obtaining an image to be detected, and performing saliency detection on the image to be detected by using at least two preset detection algorithms to obtain corresponding saliency images; performing superpixel segmentation on the image to be detected by using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and extracting features of each superpixel to obtain the image features of each superpixel; based on the image features of each saliency image and each superpixel and by using a preset difference function, determining the difference values between the background and the salient regions of each saliency image; determining the defect detection result based on the difference values of each saliency image.

[0103] In addition, when the logical instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0104] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the above-mentioned surface defect detection method, which includes: obtaining an image to be detected, and performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding saliency image; performing superpixel segmentation on the image to be detected using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and extracting features of each superpixel to obtain the image features of each superpixel; based on each saliency image and the image features of each superpixel and using a preset difference function, determining the difference value between the background and the salient region of each saliency image; and determining the defect detection result based on the difference values of each saliency image.

[0105] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the above-mentioned surface defect detection method, which includes: obtaining an image to be detected, and performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding saliency image; performing superpixel segmentation on the image to be detected using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and extracting features of each superpixel to obtain the image features of each superpixel; based on each saliency image and the image features of each superpixel and using a preset difference function, determining the difference value between the background and the salient region of each saliency image; and determining the defect detection result based on the difference values of each saliency image.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.

[0107] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A surface defect detection method, characterized in that, Including: Obtain the image to be detected, and perform saliency detection on the image to be detected by using at least two preset detection algorithms to obtain the corresponding saliency image; Perform superpixel segmentation on the image to be detected by using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and extract features of each of the superpixels to obtain the image features of each of the superpixels; Based on each of the saliency images and the image features of each of the superpixels and by using a preset difference function, determine the difference value between the background and the saliency region of each of the saliency images; Determine the defect detection result based on the difference values of each of the saliency images; Among them, the step of determining the difference value between the background and the saliency region of each of the saliency images based on each of the saliency images and the image features of each of the superpixels and by using a preset difference function includes: Determine the superpixels of the background region and the superpixels of the saliency region based on the pixel values of each of the saliency images and the pixel values of each of the superpixels; Based on the image features of each of the superpixels of the background region and the image features of each of the superpixels of the saliency region, determine the difference value between the background and the saliency region of each of the saliency images; The preset difference function is: Among them, is a preset difference degree function, is the saliency image, I is the image to be detected, is the maximum mean difference calculation function, and are the feature sets of the superpixels in the background region and the feature sets of the superpixels in the saliency region respectively, and are the numbers of the superpixels in the background region and the numbers of the superpixels in the saliency region respectively, is a radial basis kernel function, and are the image features of the i-th superpixel in the background region, and are the image features of the j-th superpixel in the saliency region; The step of determining the defect detection result based on the difference values of each of the saliency images includes: Select the saliency image corresponding to the maximum difference value among the difference values of each of the saliency images as the target saliency image; If the saliency region exists in the target saliency image, determine that the defect detection result corresponding to the image to be detected is that there is a defect; If the saliency region does not exist in the target saliency image, determine that the defect detection result corresponding to the image to be detected is that there is no defect.

2. The surface defect detection method according to claim 1, wherein Performing saliency detection on the image to be detected by using at least two preset detection algorithms to obtain the corresponding saliency image includes: Performing saliency detection on the image to be detected by using at least two preset detection algorithms to obtain the corresponding initial saliency images; Perform adaptive segmentation on each of the initial saliency images to obtain each of the saliency images.

3. The surface defect detection method according to claim 1, wherein The step of determining the superpixels of the background region and the superpixels of the saliency region based on the pixel values of each of the saliency images and the pixel values of each of the superpixels includes: For any saliency image, define the pixel points with pixel values of a first preset value among the pixel points of the saliency image as background pixel points, and define the pixel points with pixel values of a second preset value among the pixel points of the saliency image as saliency pixel points; For any superpixel, in the case where the background pixel points in the superpixel exceed a preset proportion, define the superpixel as the superpixel of the background region, and in the case where the saliency pixel points in the superpixel exceed a preset proportion, define the superpixel as the superpixel of the saliency region.

4. The surface defect detection method according to any one of claims 1-3, characterized in that, The image features of the superpixel at least include the mean value of the gray percentage, the image entropy, and the Gabor feature.

5. A surface defect detection device, characterized in that, Including: A saliency detection module, configured to obtain the image to be detected, and perform saliency detection on the image to be detected by using at least two preset detection algorithms to obtain the corresponding saliency image; A superpixel segmentation module, configured to perform superpixel segmentation on the image to be detected by using a preset superpixel segmentation algorithm, obtain a plurality of superpixels, and extract features of each of the superpixels to obtain the image features of each of the superpixels; A difference degree calculation module, configured to determine the difference degree values between the background and the significant regions of each of the significant images based on each of the significant images and the image features of each of the superpixels and by using a preset difference degree function; A result determination module, configured to determine a defect detection result based on the difference degree values of each of the significant images; The difference degree calculation module is further configured to determine background region superpixels and significant region superpixels based on the pixel values of each of the significant images and the pixel values of each of the superpixels; and determine the difference degree values between the background and the significant regions of each of the significant images based on the image features of each of the background region superpixels and the image features of each of the significant region superpixels; The result determination module is further configured to select, as a target significant image, the significant image corresponding to the maximum difference degree value among the difference degree values of each of the significant images; if the significant region exists in the target significant image, determine that the defect detection result corresponding to the image to be detected is that there is a defect; if the significant region does not exist in the target significant image, determine that the defect detection result corresponding to the image to be detected is that there is no defect; The preset difference degree function is: Among them, is a preset difference degree function, is the saliency image, I is the image to be detected, is the maximum mean difference calculation function, and are the feature sets of the superpixels in the background region and the feature sets of the superpixels in the saliency region respectively, and are the numbers of the superpixels in the background region and the numbers of the superpixels in the saliency region respectively, is a radial basis kernel function, and are the image features of the i-th superpixel in the background region, and are the image features of the j-th superpixel in the saliency region.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the surface defect detection method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the surface defect detection method according to any one of claims 1 to 4 is implemented.

Images