Surface defect detection method, device and equipment and storage medium

Abstract

The invention discloses a surface defect detection method and device, equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a to-be-detected image, and performing saliency detection on the to-be-detected image by using at least two preset detection algorithms to obtain a corresponding saliency image; performing superpixel segmentation on the to-be-detected image by using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and performing feature extraction on each superpixel to obtain an image feature of each superpixel; determining a difference degree value between the background of each saliency image and the saliency region by using a preset difference degree function based on the image features of each saliency image and each superpixel; and determining a defect detection result based on the difference degree value of each saliency image. Therefore, the precision of surface defect detection is improved, and the problem that different images are difficult to accurately adapt to the saliency detection algorithm with the optimal performance is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a surface defect detection method, device, equipment and storage medium. Background Art

[0002] Saliency detection technology is mainly used to find image areas that can arouse human visual cognitive interest. It is the basis of various tasks in computer vision, such as image cropping, image compression, image stitching, image classification and target recognition.

[0003] However, due to the complexity of the perception mechanism of the human visual system, there are currently a variety of saliency detection algorithms designed based on different perspectives; in addition, different images contain complex and diverse visual patterns, so different saliency detection algorithms are often adapted for different images. For example, for the defect detection task on the aircraft surface, due to the large curved surface, complex structure, and high manufacturing precision of the aircraft titanium alloy body, the probability of surface defects is low, the image metal texture is complex, the defect area is small, and the background-defect contrast is low due to the dim metal luster. Therefore, it is impossible to accurately detect defects on the aircraft surface using the existing saliency detection technology. Summary of the invention

[0004] The main purpose of this application is to provide a surface defect detection method, device, equipment and storage medium to improve the accuracy of surface defect detection and solve the problem that different images are difficult to accurately adapt to the saliency detection algorithm with the best performance.

[0005] To achieve the above objectives, the present application provides a surface defect detection method, comprising: Acquire an image to be detected, and perform saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding saliency image; Performing superpixel segmentation on the image to be detected using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and performing feature extraction on each of the superpixels to obtain image features of each of the superpixels; Based on the image features of each saliency image and each superpixel and using a preset difference function, determining a difference value between the background and the saliency region of each saliency image; The defect detection result is determined based on the difference value of each of the saliency images.

[0006] Optionally, performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding saliency image includes: performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding initial saliency image; and adaptively segmenting each of the initial saliency images to obtain each of the saliency images.

[0007] Optionally, the method of determining the difference value between the background and the salient area of ​​each salient image based on the image features of each salient image and each superpixel and using a preset difference function includes: determining the background area superpixels and the salient area superpixels based on the pixel values ​​of each salient image and the pixel values ​​of each superpixel; and determining the difference value between the background and the salient area of ​​each salient image based on the image features of each background area superpixel and the image features of each salient area superpixel.

[0008] Optionally, the determining of background area superpixels and salient area superpixels based on the pixel values ​​of each of the salient images and the pixel values ​​of each of the superpixels includes: for any salient image, defining the pixel points whose pixel values ​​of each pixel points in the salient image are a first preset value as background pixels, and defining the pixel points whose pixel values ​​of each pixel points in the salient image are a second preset value as salient pixels; for any superpixel, when the background pixel points in the superpixel exceed a preset proportion, defining the superpixel as the background area superpixel, and when the salient pixel points in the superpixel exceed a preset proportion, defining the superpixel as the salient area superpixel.

[0009] Optionally, the preset difference function is:

[0010] in, is the preset difference function, is the saliency image, I is the image to be detected, is the maximum mean difference calculation function, and are the feature set of the background area superpixel and the feature set of the salient area superpixel, respectively. and are the number of superpixels in the background area and the number of superpixels in the salient area, respectively. is the radial basis kernel function, and is the image feature of the i-th background area superpixel, and is the image feature of the jth superpixel in the salient region.

[0011] Optionally, determining the defect detection result based on the difference values ​​of each of the saliency images includes: selecting a saliency image corresponding to the largest difference value among the difference values ​​of each of the saliency images as a target saliency image; if the target saliency image has the saliency area, determining that the defect detection result corresponding to the image to be detected is that a defect exists; if the target saliency image does not have the saliency area, determining that the defect detection result corresponding to the image to be detected is that a defect does not exist.

[0012] Optionally, the image features of the superpixel include at least grayscale percentage mean, image entropy and Gabor features.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a surface defect detection device, including: a saliency detection module, used to obtain an image to be detected, and use at least two preset detection algorithms to perform saliency detection on the image to be detected to obtain a corresponding saliency image; a superpixel segmentation module, used to use a preset superpixel segmentation algorithm to perform superpixel segmentation on the image to be detected to obtain multiple superpixels, and perform feature extraction on each of the superpixels to obtain image features of each superpixel; a difference calculation module, used to determine the difference value between the background and the saliency area of ​​each saliency image based on the image features of each saliency image and each superpixel and using a preset difference function; a result determination module, used to determine the defect detection result based on the difference value of each saliency image.

[0014] The present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the surface defect detection methods described above when executing the computer program.

[0015] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the surface defect detection method as described in any one of the above is implemented.

[0016] The surface defect detection method of the present application performs saliency detection on an image to be detected by using a plurality of preset detection algorithms to obtain saliency images corresponding to each preset detection algorithm; then uses a preset superpixel segmentation algorithm to perform superpixel segmentation on the image to be detected to obtain a plurality of superpixels, and performs feature extraction on each superpixel to obtain image features of each superpixel; further based on the image features of each saliency image and each superpixel and using a preset difference function, determines the difference value between the background and the saliency area of ​​each saliency image; finally, determines the defect detection result based on the difference value of each saliency image, and integrates the detection results of a plurality of saliency detection algorithms with the result of the superpixel segmentation algorithm, so as to be able to match the saliency detection algorithm with the best performance for the image to be detected, thereby improving the accuracy of surface defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is one of the flow charts of the surface defect detection method of an embodiment of the present application; FIG2 (a) is an original image of an image to be detected in a specific example of the present application; FIG2( b ) is an initial saliency image output by a BC algorithm model of a specific example of the present application; FIG2( c ) is an initial saliency image output by the FT algorithm model of a specific example of the present application; FIG2( d ) is an initial saliency image output by a GS algorithm model of a specific example of the present application; FIG2( e ) is an initial saliency image output by the MCITF algorithm model of a specific example of the present application; FIG3 (a) is a result diagram of an initial saliency image after adaptive segmentation corresponding to a specific example of the BC algorithm of the present application; FIG3( b ) is a result diagram of an initial saliency image after adaptive segmentation corresponding to the FT algorithm of a specific example of the present application; FIG3 (c) is a result diagram of an initial saliency image after adaptive segmentation corresponding to the GS algorithm of a specific example of the present application; FIG3(d) is a result diagram of an initial saliency image after adaptive segmentation corresponding to the MCITF algorithm of a specific example of the present application; Figure 4 is a schematic diagram of a superpixel segmentation result of a specific example of the present application; Figure 5 This is the second flow chart of the surface defect detection method of the embodiment of the present application; Figure 6 This is the third flow chart of the surface defect detection method of the embodiment of the present application; Figure 7 is a schematic structural diagram of a surface defect detection device according to an embodiment of the present application; Figure 8 An example of a physical structure diagram of an electronic device is provided; In the figure, 700, surface defect detection device; 710, saliency detection module; 720, superpixel segmentation module; 730, difference calculation module; 740, result determination module; 810, processor; 820, communication interface; 830, memory; 840, communication bus.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] The saliency detection algorithm imitates the human visual system and detects salient areas by analyzing the differences between different areas of the image. However, due to the complexity of the perception mechanism of the human visual system, existing methods are designed from different angles. This theoretical design difference leads to certain differences in the detection results of different algorithms when detecting the same image. In addition, different images contain complex and diverse visual patterns, so different images are often adapted to different saliency detection algorithms.

[0021] Especially in the application scenario of aircraft body surface defect detection, due to the characteristics of aircraft titanium alloy body such as large curved surface, complex structure and high manufacturing precision, the probability of surface defects is low, the metal texture of the image is complex, the defect area is small, and the background-defect contrast is low due to the dull metal gloss.

[0022] However, the existing saliency detection algorithms mainly detect surface defects by generating saliency maps. In addition, the existing saliency detection algorithms need to collect training samples in advance and provide benchmark true values ​​to train the classifier. At the same time, they only use a single saliency model to judge the presence or absence of defects, lacking multi-angle and multi-dimensional information. Therefore, the detection accuracy of aircraft body surface defects is low and the model training is complex if the existing saliency detection algorithms are used alone.

[0023] Based on this, the embodiments of the present application provide a surface defect detection method, device, equipment and storage medium, which integrate multiple saliency detection algorithms and comprehensively consider multi-angle and multi-dimensional information to improve the accuracy of surface defect detection; and the embodiments of the present application do not need to collect training samples to provide benchmark true values, so they are more suitable for small sample scenarios, and solve the problem that in the face of the wide variety of existing saliency detection algorithms, different images are difficult to accurately adapt to the saliency detection algorithm with the best performance.

[0024] Figure 1 is one of the flow charts of the surface defect detection method of the embodiment of the present application. The surface defect detection method can be executed by a processor of any electronic device, such as Figure 1 As shown, the surface defect detection method may include the following steps: Step 110: Acquire an image to be detected, and use at least two preset detection algorithms to perform saliency detection on the image to be detected to obtain a corresponding saliency image.

[0025] Step 120: Use a preset superpixel segmentation algorithm to perform superpixel segmentation on the image to be detected to obtain multiple superpixels, and perform feature extraction on each superpixel to obtain image features of each superpixel.

[0026] Step 130: Based on the image features of each saliency image and each superpixel and using a preset difference function, determine the difference value between the background and the saliency region of each saliency image.

[0027] Step 140: Determine the defect detection result based on the difference value of each saliency image.

[0028] First of all, it should be noted that the embodiments of the present application can be applied to any metal surface defect detection scenario, and in particular, can be applied to the scenario of titanium alloy metal surface defect detection for aircraft bodies. The defect rate of titanium alloy metal processing is very low. If traditional deep learning-based feature extraction and fusion methods are used, it is difficult to collect sufficient training data. Therefore, the scenario of titanium alloy metal surface defect detection for aircraft bodies is a typical small sample scenario, and the benchmark true value (i.e., the significant area containing surface defects) often requires manual annotation, which is relatively costly.

[0029] In step 110, the image to be detected can be captured by a camera. After the image to be detected is obtained, the image to be detected can be simultaneously input into multiple preset detection algorithms. Each preset detection algorithm performs saliency detection on the image to be detected to obtain a saliency image corresponding to each preset detection algorithm.

[0030] In this embodiment, multiple preset detection algorithms may include (but are not limited to): MCITF saliency detection algorithm (Saliency detection for strip steel surface defects using multiple constraints and improved texture features), BC saliency detection algorithm (Saliency Optimization from Robust Background Detection), FT saliency detection algorithm (Frequency-Tuned Salient Region Detection), and GS saliency detection algorithm (Geodesic Saliency Using Background Prior).

[0031] The following uses the above four saliency detection algorithms as preset detection algorithms as an example to introduce the surface defect detection method of the embodiment of the present application in detail.

[0032] In some embodiments, performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding saliency image in step 110 may include: performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding initial saliency image; and adaptively segmenting each initial saliency image to obtain each saliency image.

[0033] Specifically, after obtaining the image to be detected, the image to be detected can be simultaneously input into the MCITF algorithm model, the BC algorithm model, the FT algorithm model and the GS algorithm model. The algorithm models of the preset detection algorithms respectively perform saliency detection on the image to be detected and output the corresponding initial saliency image. In this embodiment, the detection results output by the above four preset detection algorithms can be recorded as: , , , .

[0034] Figure 2 (a) is the original image of the image to be detected in a specific example of the present application; Figure 2 (b) is the initial saliency image output by the BC algorithm model in a specific example of the present application; Figure 2 (c) is the initial saliency image output by the FT algorithm model in a specific example of the present application; Figure 2 (d) is the initial saliency image output by the GS algorithm model in a specific example of the present application; Figure 2 (e) is the initial saliency image output by the MCITF algorithm model in a specific example of the present application. It can be seen from Figures 2 (a) to 2 (e) that the saliency area and background area in the image to be detected are distinguished by the above four preset detection models, but different saliency detection algorithms output different images of saliency detection results when performing saliency detection on the same image to be detected.

[0035] It should be noted that the structures and detection processes of the above four saliency detection algorithms can refer to the structures and detection processes of existing saliency detection algorithms, which will not be described in detail here.

[0036] After obtaining the initial saliency images output by each preset detection algorithm, each initial saliency image can be adaptively segmented to obtain each saliency image. Specifically, the maximum inter-class variance method can be used to binarize the above calculation results respectively, and all the processing results are recorded as , .

[0037] It should be noted that the maximum inter-class variance method finds a suitable threshold by traversing the grayscale histogram of the initial saliency image; and then compares the pixel value of each pixel in the initial saliency image with the threshold to divide each pixel in the initial saliency image into a pixel whose pixel value is a first preset value and a pixel whose pixel value is a second preset value.

[0038] As an example, first process the input initial saliency image to obtain its grayscale histogram. This histogram describes the frequency of occurrence of different grayscale levels in the image. Further traverse each possible grayscale as a candidate threshold. For each candidate threshold, divide all pixels in the image into two categories: the first category (foreground): all pixels with pixel values ​​greater than or equal to the threshold; the second category (background): all pixels with pixel values ​​less than the threshold. Next, calculate the intra-class variance and inter-class variance: for each category, calculate their mean and their respective intra-class variance (that is, the degree of dispersion of the pixel value distribution within the class). Then calculate the inter-class variance, which is the weighted sum of the square of the difference between the means of the two categories and the proportion of each occupying the image. Select the threshold that maximizes the inter-class variance as the final segmentation threshold. The purpose of this step is to ensure that there is the greatest difference between the two parts (foreground and background) after segmentation. Use the best threshold found to binarize the original image. For each pixel in the image, its pixel value is compared with the optimal threshold: if the pixel value is less than the threshold, the pixel is marked as "background" and assigned a first preset value (usually black, i.e. 0); if the pixel value is greater than or equal to the threshold, the pixel is marked as "foreground" and assigned a second preset value (usually white, i.e. 255).

[0039] Figure 3 (a) is a result diagram of the BC algorithm of a specific example of the present application corresponding to the initial salient image after adaptive segmentation; Figure 3 (b) is a result diagram of the FT algorithm of a specific example of the present application corresponding to the initial salient image after adaptive segmentation; Figure 3 (c) is a result diagram of the GS algorithm of a specific example of the present application corresponding to the initial salient image after adaptive segmentation; Figure 3 (d) is a result diagram of the MCITF algorithm of a specific example of the present application corresponding to the initial salient image after adaptive segmentation. It can be seen from Figures 3 (a) to 3 (d) that after the initial salient image is adaptively segmented, all pixels are divided into two types according to pixel values, and the salient area and the background area can be clearly seen.

[0040] In step 120, after obtaining the image to be detected, the image to be detected can be segmented into superpixels using a preset superpixel segmentation algorithm. In this embodiment, a simple linear iterative clustering algorithm can be used to segment the image to be detected into superpixels, so that the image to be detected is segmented into a plurality of superpixels according to local pixel similarities. Specifically, superpixel segmentation is based on the similarity of features such as color, brightness, texture, etc. between pixels of the image to be detected, and the image is subdivided into a plurality of small areas, which are superpixels.

[0041] Figure 4 FIG. 1 is a schematic diagram of a superpixel segmentation result of a specific example of the present application. Figure 4As shown in Figure 2, through superpixel segmentation, the image to be detected is divided into several small areas (i.e., superpixels). The pixel features of each superpixel are similar. Figure 4 From the segmentation results in , it can also be concluded which part of the superpixels is the salient area and which part of the superpixels is the background area. It should be noted that the preset superpixel segmentation algorithm can adopt the existing superpixel segmentation algorithm, such as the K-means algorithm, the Mean Shift algorithm, etc. The specific process of superpixel segmentation can also refer to the existing superpixel segmentation process, which will not be repeated here.

[0042] Furthermore, after the superpixel segmentation is performed to obtain a plurality of superpixels, feature extraction can be performed on each superpixel to obtain the image features of each superpixel. In some embodiments, the image features of the superpixel at least include the grayscale percentage mean , Image Entropy and Gabor features.

[0043] Among them, the grayscale percentage mean means: for a given superpixel area, the average grayscale value of all pixels is calculated. This can be used to characterize the overall brightness level of the superpixel. Secondly, image entropy can measure the uncertainty or complexity of information in the image. For a superpixel, entropy can be calculated based on the grayscale histogram of its internal pixels. The higher the entropy, the greater the grayscale variation within the superpixel and the higher the information content. Finally, Gabor features are features extracted based on Gabor filters; these filters can capture information of different directions and frequencies in the image. For a superpixel, texture information can be extracted by applying Gabor filters to the area and calculating the response strength. Commonly used Gabor features include response strength in directionality and scale. The specific image feature extraction process can refer to the existing image feature extraction process, which will not be repeated here.

[0044] In addition, it should be noted that the order of step 110 and step 120 can be swapped. In short, the image to be detected can be first segmented into superpixels using a preset superpixel segmentation algorithm, and then the image to be detected can be subjected to saliency detection using a variety of preset detection algorithms. This embodiment only provides an example process of a surface defect detection method and should not limit the scope of protection of this application.

[0045] Furthermore, in step 130, after obtaining the saliency images corresponding to each preset detection algorithm and the image features of each superpixel, a pre-established preset difference function can be used to perform weight calculations on the saliency images corresponding to each preset detection algorithm, thereby being able to quantitatively evaluate the detection results of each preset detection algorithm, so as to subsequently select the most accurate detection result from all saliency detection results.

[0046] Figure 5 This is the second flow chart of the surface defect detection method of the embodiment of the present application. Figure 5 As shown, in some embodiments, step 130 determines the difference value between the background and the salient area of ​​each salient image based on the image features of each salient image and each superpixel and using a preset difference function, which may include the following steps: Step 510: Determine background region superpixels and salient region superpixels based on the pixel values ​​of each salient image and the pixel values ​​of each superpixel.

[0047] Step 520: Determine the difference value between the background and the salient area of ​​each salient image based on the image features of the superpixels of each background area and the image features of the superpixels of each salient area.

[0048] In this embodiment, because each saliency image has been binarized, the background area and the saliency area can be distinguished based on the pixel value of the saliency image. The embodiment of the present application combines the binarization result of the saliency image with the superpixel segmentation result to obtain the corresponding background area and saliency area.

[0049] It should be noted that the background area and salient area determined by using different binarization processing results of salient images may also be different.

[0050] In some embodiments, determining background area superpixels and salient area superpixels based on the pixel values ​​of each salient image and the pixel values ​​of each superpixel may include: for any salient image, defining a pixel whose pixel value in the salient image is a first preset value as a background pixel, and defining a pixel whose pixel value in the salient image is a second preset value as a salient pixel; for any superpixel, if the background pixel in the superpixel exceeds a preset proportion, the superpixel is defined as a background area superpixel, and if the salient pixel in the superpixel exceeds a preset proportion, the superpixel is defined as a salient area superpixel.

[0051] It should be noted that the first preset value, the second preset value and the preset ratio can be set according to actual needs. For example, the first preset value can be set to 0, the second preset value can be set to 1, and the preset ratio can be set to 50%. The first preset value, the second preset value and the preset ratio are not specifically limited here.

[0052] The following takes the first preset value as 0, the second preset value as 1, and the preset proportion as 50% as an example to describe this step in detail.

[0053] Taking a saliency image as an example, the pixel points with a pixel value of 0 in the saliency image after binary processing can be defined as background pixels, and the pixel points with a pixel value of 1 in the saliency image after binary processing can be defined as saliency pixels. Then the definition results of each pixel in the above saliency image are applied to superpixels. For a certain superpixel, if the proportion of background pixels in the superpixel exceeds 50%, the superpixel can be defined as a background area superpixel; if the proportion of saliency pixels in the superpixel exceeds 50%, the superpixel can be defined as a saliency area superpixel. In this way, superpixels and saliency images can be combined to determine background area superpixels and saliency area superpixels.

[0054] The background region superpixels and salient region superpixels corresponding to each salient image can be determined in the same manner. Furthermore, after the background region superpixels and salient region superpixels corresponding to each salient image are determined, the difference value between the background and the salient region can be calculated using a preset difference function. It can be understood that if the difference value between the background and the salient region is larger, it means that the difference between the salient region and the surrounding background is more obvious, and thus it is easier to be perceived or detected. At this time, it is considered that the segmentation effect of the salient image is the best.

[0055] In some implementations, the preset difference function may be:

[0056] in, is the preset difference function, is the salient image, I is the image to be detected, is the maximum mean difference calculation function, and are the feature set of the background area superpixel and the feature set of the salient area superpixel, respectively. and are the number of superpixels in the background area and the number of superpixels in the salient area, respectively. is the radial basis kernel function, and is the image feature of the superpixel in the i-th background area, and is the image feature of the j-th salient region superpixel.

[0057] After determining the background area superpixels and salient area superpixels corresponding to each salient image and the image features of each superpixel, the above parameters can be substituted into the preset difference function to obtain the background-salient area difference values ​​corresponding to each preset detection algorithm.

[0058] In step 140 , after obtaining the difference values ​​of the background-salient area corresponding to each preset detection algorithm, the defect detection result can be determined based on the difference values ​​of each salient image. Figure 6 This is the third flow chart of the surface defect detection method of the embodiment of the present application. Figure 6 As shown, in some embodiments, determining the defect detection result based on the difference value of each saliency image may include the following steps: Step 610: Select the saliency image corresponding to the largest difference value among the difference values ​​of the saliency images as the target saliency image.

[0059] Step 620: If the target salient image has a salient area, then determine that the defect detection result corresponding to the image to be detected is that there is a defect.

[0060] Step 630: If the target salient image does not have a salient area, then determining that the defect detection result corresponding to the image to be detected is that there is no defect.

[0061] Specifically, after calculating the difference values ​​of each salient image, all the difference values ​​can be compared, and the salient image with the largest difference value is used as the target salient image. In this embodiment, step 610 can be implemented using the following formula:

[0062] in, is the difference value of the target saliency image finally output, , , , They are the background-salient region difference values ​​of the salient image corresponding to the MCITF algorithm, FT algorithm, BC algorithm and GS algorithm respectively.

[0063] Furthermore, after determining the target saliency image with the best segmentation effect, it is possible to determine whether there is a defect in the image to be detected by judging whether there is a saliency region in the target saliency image. Specifically, if there is a region with a pixel value of the second preset value in the target saliency image, it can be determined that there is a defect in the image to be detected, and the region with a pixel value of the second preset value is the defect; if there is no region with a pixel value of the second preset value in the target saliency image, it can be determined that there is no defect in the image to be detected.

[0064] Therefore, compared with the traditional significant surface defect detection algorithm, the embodiment of the present application can significantly improve the accuracy of surface defect detection by integrating multiple significant detection algorithms and comprehensively considering multi-angle and multi-dimensional information; secondly, the defect detection method of the embodiment of the present application is highly scalable. If a new significant detection algorithm appears, the new significant detection algorithm can be directly added to the method; finally, the defect detection method provided by the embodiment of the present application does not need to collect training samples or provide benchmark true values, and is more suitable for small sample scenarios similar to aircraft body defect detection scenarios.

[0065] Based on the above embodiments, the embodiments of the present application also provide a surface defect detection device. Figure 7 Schematic diagram of the structure of the surface defect detection device of the embodiment of the present application. Figure 7 As shown, the surface defect detection device 700 may include: a saliency detection module 710, a superpixel segmentation module 720, a difference calculation module 730 and a result determination module 740, and the saliency detection module 710, the superpixel segmentation module 720, and the result determination module 740 are respectively connected to the difference calculation module 730.

[0066] Among them, the saliency detection module 710 is used to obtain the image to be detected, and use at least two preset detection algorithms to perform saliency detection on the image to be detected to obtain the corresponding saliency image; the superpixel segmentation module 720 is used to use the preset superpixel segmentation algorithm to perform superpixel segmentation on the image to be detected to obtain multiple superpixels, and perform feature extraction on each superpixel to obtain the image feature of each superpixel; the difference calculation module 730 is used to determine the difference value between the background and the saliency area of ​​each saliency image based on the image features of each saliency image and each superpixel and using a preset difference function; the result determination module 740 is used to determine the defect detection result based on the difference value of each saliency image.

[0067] Therefore, the saliency detection module 710 performs saliency detection on the image to be detected by using multiple preset detection algorithms to obtain saliency images corresponding to each preset detection algorithm; the superpixel segmentation module 720 then uses the preset superpixel segmentation algorithm to perform superpixel segmentation on the image to be detected to obtain multiple superpixels, and extracts features from each superpixel to obtain image features of each superpixel; further, the difference calculation module 730 determines the difference value between the background and the saliency area of ​​each saliency image based on the image features of each saliency image and each superpixel and using a preset difference function; finally, the result determination module 740 determines the defect detection result based on the difference value of each saliency image, and integrates the detection results of multiple saliency detection algorithms with the results of the superpixel segmentation algorithm, so as to match the saliency detection algorithm with the best performance for the image to be detected, thereby improving the accuracy of surface defect detection.

[0068] In some embodiments, the saliency detection module 710 is specifically used to: perform saliency detection on the image to be detected using at least two preset detection algorithms to obtain corresponding initial saliency images; and perform adaptive segmentation on each initial saliency image to obtain each saliency image.

[0069] In some embodiments, the difference calculation module 730 is specifically used to: determine the background area superpixels and the salient area superpixels based on the pixel values ​​of each salient image and the pixel values ​​of each superpixel; determine the difference value between the background and the salient area of ​​each salient image based on the image features of each background area superpixel and the image features of each salient area superpixel.

[0070] In some embodiments, the difference calculation module 730 is also specifically used to: for any saliency image, define the pixel points whose pixel values ​​in the saliency image are the first preset value as background pixels, and define the pixel points whose pixel values ​​in the saliency image are the second preset value as saliency pixels; for any superpixel, when the background pixels in the superpixel exceed the preset proportion, define the superpixel as a background area superpixel, and when the saliency pixels in the superpixel exceed the preset proportion, define the superpixel as a saliency area superpixel.

[0071] In some implementations, the preset difference function is:

[0072] in, is the preset difference function, is the salient image, I is the image to be detected, is the maximum mean difference calculation function, and are the feature set of the background area superpixel and the feature set of the salient area superpixel, respectively. and are the number of superpixels in the background area and the number of superpixels in the salient area, respectively. is the radial basis kernel function, and is the image feature of the superpixel in the i-th background area, and is the image feature of the j-th salient region superpixel.

[0073] In some embodiments, the result determination module 740 is specifically used to: select the saliency image corresponding to the largest difference value among the difference values ​​of each saliency image as the target saliency image; if the target saliency image has a saliency area, determine that the defect detection result corresponding to the image to be detected is that there is a defect; if the target saliency image does not have a saliency area, determine that the defect detection result corresponding to the image to be detected is that there is no defect.

[0074] In some embodiments, the image features of the superpixel include at least grayscale percentage mean, image entropy, and Gabor features.

[0075] It should be noted that for details not disclosed in the surface defect detection device of this embodiment, please refer to the details disclosed in the embodiment of the surface defect detection method in the embodiments of this specification, and no further details will be given here.

[0076] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the surface defect detection method, which includes: acquiring an image to be detected, and performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding saliency image; performing superpixel segmentation on the image to be detected using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and performing feature extraction on each superpixel to obtain image features of each superpixel; determining the difference value between the background and the saliency area of ​​each saliency image based on the image features of each saliency image and each superpixel and using a preset difference function; and determining the defect detection result based on the difference value of each saliency image.

[0077] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0078] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the above-mentioned surface defect detection method, which includes: acquiring an image to be detected, and using at least two preset detection algorithms to perform saliency detection on the image to be detected to obtain a corresponding saliency image; using a preset superpixel segmentation algorithm to perform superpixel segmentation on the image to be detected to obtain multiple superpixels, and performing feature extraction on each superpixel to obtain image features of each superpixel; based on the image features of each saliency image and each superpixel and using a preset difference function, determining the difference value between the background and the saliency area of ​​each saliency image; determining the defect detection result based on the difference value of each saliency image.

[0079] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the above-mentioned surface defect detection method, the method comprising: acquiring an image to be detected, and performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding saliency image; performing superpixel segmentation on the image to be detected using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and performing feature extraction on each superpixel to obtain image features of each superpixel; determining the difference value between the background and the saliency area of ​​each saliency image based on the image features of each saliency image and each superpixel and using a preset difference function; and determining a defect detection result based on the difference value of each saliency image.

[0080] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A surface defect detection method, characterized in that: include: Acquire an image to be detected, and perform saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding saliency image; Performing superpixel segmentation on the image to be detected by using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and performing feature extraction on each of the superpixels to obtain image features of each of the superpixels; Based on the image features of each saliency image and each superpixel and using a preset difference function, determining a difference value between the background and the saliency region of each saliency image; The defect detection result is determined based on the difference value of each of the saliency images.

2. The surface defect detection method according to claim 1, characterized in that: Performing saliency detection on the image to be detected by using at least two preset detection algorithms to obtain a corresponding saliency image, including: Performing saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding initial saliency image; Adaptively segmenting each of the initial saliency images to obtain each of the saliency images.

3. The surface defect detection method according to claim 1, characterized in that: The determining of the difference value between the background and the salient area of ​​each salient image based on the image features of each salient image and each superpixel and using a preset difference function comprises: Determining background area superpixels and salient area superpixels based on the pixel values ​​of each of the saliency images and the pixel values ​​of each of the superpixels; Based on the image features of the superpixels of each background region and the image features of the superpixels of each salient region, a difference value between the background and the salient region of each salient image is determined.

4. The surface defect detection method according to claim 3, characterized in that: The determining of the background area superpixel and the salient area superpixel based on the pixel value of each salient image and the pixel value of each superpixel comprises: For any saliency image, a pixel point whose pixel value of each pixel point in the saliency image is a first preset value is defined as a background pixel point, and a pixel point whose pixel value of each pixel point in the saliency image is a second preset value is defined as a saliency pixel point; For any superpixel, when the background pixel points in the superpixel exceed a preset proportion, the superpixel is defined as the background area superpixel; when the significant pixel points in the superpixel exceed a preset proportion, the superpixel is defined as the significant area superpixel.

5. The surface defect detection method according to claim 4, characterized in that: The preset difference function is: in, is the preset difference function, is the saliency image, I is the image to be detected, is the maximum mean difference calculation function, and are the feature set of the background area superpixel and the feature set of the salient area superpixel, respectively. and are the number of superpixels in the background area and the number of superpixels in the salient area, respectively. is the radial basis kernel function, and is the image feature of the i-th background area superpixel, and is the image feature of the jth superpixel in the salient region.

6. The surface defect detection method according to claim 3, characterized in that: The step of determining the defect detection result based on the difference value of each saliency image comprises: Selecting a saliency image corresponding to the largest difference value among the difference values ​​of the saliency images as a target saliency image; If the target salient image has the salient area, determining that the defect detection result corresponding to the image to be detected is that a defect exists; If the target salient image does not have the salient area, it is determined that the defect detection result corresponding to the image to be detected is that there is no defect.

7. The surface defect detection method according to any one of claims 1 to 6, characterized in that: The image features of the superpixel include at least grayscale percentage mean, image entropy and Gabor features.

8. A surface defect detection device, characterized in that: include: A saliency detection module, used to obtain an image to be detected, and perform saliency detection on the image to be detected using at least two preset detection algorithms to obtain a corresponding saliency image; A superpixel segmentation module is used to perform superpixel segmentation on the image to be detected using a preset superpixel segmentation algorithm to obtain a plurality of superpixels, and perform feature extraction on each of the superpixels to obtain image features of each of the superpixels; A difference calculation module, used to determine the difference value between the background and the salient area of ​​each salient image based on the image features of each salient image and each superpixel and using a preset difference function; The result determination module is used to determine the defect detection result based on the difference value of each of the saliency images.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the surface defect detection method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the surface defect detection method according to any one of claims 1 to 7 is implemented.

Images