Defect detection method and device, electronic equipment and computer readable storage medium

Abstract

The application provides a defect detection method and device, electronic equipment and a computer readable storage medium. The method comprises the steps of: obtaining a target detection image of a target workpiece, and pre-processing the target detection image to obtain a pre-processed image; performing defect detection on the target detection image and the pre-processed image through a trained defect detection model, wherein the target detection image and the pre-processed image are subjected to feature extraction at a feature extraction level of the trained defect detection model to obtain joint features; and outputting a defect detection result based on the joint features obtained at each level through the trained defect detection model. The joint features can reflect the original defect condition of the target detection image and the defect condition after enhancement of the pre-processed image, thereby reducing the influence of image problems on defect detection and improving detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a defect detection method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] In existing workpiece defect detection, such as defect detection on X-ray images of steering knuckle castings, some defects in the image have low contrast, uneven gray scale distribution or blurred boundaries, and other image problems, which are difficult to be accurately identified by existing defect detection methods, and the detection accuracy is low. SUMMARY

[0003] The main purpose of the present application is to provide a defect detection method, device, electronic equipment and computer readable storage medium, which aims to solve the problem of low defect detection accuracy in the prior art.

[0004] To achieve the above purpose, the present application provides a defect detection method, which comprises the following steps:

[0005] Obtain a target detection image of a target workpiece, and pre-process the target detection image to obtain a pre-processed image;

[0006] The target detection image and the pre-processed image are subjected to defect detection by a trained defect detection model, wherein the target detection image and the pre-processed image are subjected to feature extraction at a feature extraction level of the trained defect detection model to obtain joint features;

[0007] The trained defect detection model outputs a defect detection result based on the joint features obtained at each level.

[0008] Optionally, the pre-processing of the target detection image to obtain a pre-processed image comprises:

[0009] Obtain the original contrast of the target detection image;

[0010] Obtain a contrast threshold and determine whether the original contrast is less than the contrast threshold;

[0011] If the original contrast is less than the contrast threshold, the target detection image is subjected to contrast enhancement to obtain the pre-processed image, wherein the contrast of the pre-processed image is greater than or equal to the contrast threshold.

[0012] Optionally, the pre-processing of the target detection image to obtain a pre-processed image comprises:

[0013] Obtain the original gray scale variance of the target detection image;

[0014] acquire a gray scale variance threshold value, and determine whether the original gray scale variance is greater than the gray scale variance threshold value;

[0015] If the original gray scale variance is greater than the gray scale variance threshold value, eliminate the local gray scale difference of the target detection image to obtain the preprocessed image.

[0016] Optionally, the acquiring of the contrast threshold value comprises:

[0017] acquire a first historical missed detection image, and determine a historical contrast corresponding to each of the first historical missed detection images;

[0018] take the maximum historical contrast as a reference contrast;

[0019] determine the contrast threshold value according to the reference contrast, wherein the contrast threshold value is greater than the reference contrast.

[0020] Optionally, the feature extraction of the target detection image and the preprocessed image to obtain the joint feature comprises:

[0021] extract features of the target detection image to obtain original image features;

[0022] extract features of the preprocessed image to obtain preprocessed features;

[0023] acquire a proportion coefficient, and perform feature fusion on the original image features and the preprocessed features based on the proportion coefficient to obtain the joint feature.

[0024] Optionally, the defect detection of the target detection image and the preprocessed image by the trained defect detection model comprises:

[0025] acquire an initial defect detection model, wherein the defect detection model comprises an auxiliary decoding head, and a hole spatial pyramid pooling layer is arranged in the auxiliary decoding head;

[0026] acquire a defect detection training sample, and train the initial defect detection model by using the defect detection training sample to obtain the trained defect detection model.

[0027] Optionally, the defect detection model comprises a dropout layer, and the defect detection of the target detection image and the preprocessed image by the trained defect detection model comprises:

[0028] acquire a real-time workpiece type quantity and a data set category quantity corresponding to the defect detection model;

[0029] determine a target discard probability corresponding to the number of real-time workpiece types according to the number of dataset categories, wherein the target discard probability is negatively correlated with the number of real-time workpiece types;

[0030] set a discard probability of the discard layer as the target discard probability.

[0031] To achieve the above object, the present application further provides a defect detection device, which comprises:

[0032] a first acquisition module, configured to acquire a target detection image of a target workpiece and to obtain a preprocessed image by preprocessing the target detection image;

[0033] a first detection module, configured to perform defect detection on the target detection image and the preprocessed image by using a trained defect detection model, wherein a joint feature is obtained by performing feature extraction on the target detection image and the preprocessed image at a feature extraction level of the trained defect detection model;

[0034] a first execution module, configured to output a defect detection result based on the joint feature obtained at each level by using the trained defect detection model.

[0035] To achieve the above object, the present application further provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is executed by the processor to implement the steps of the defect detection method.

[0036] To achieve the above object, the present application further provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program is executed by a processor to implement the steps of the defect detection method.

[0037] The application provides a defect detection method and device, an electronic device and a computer readable storage medium. The method comprises the following steps: obtaining a target detection image of a target workpiece, and preprocessing the target detection image to obtain a preprocessed image; performing defect detection on the target detection image and the preprocessed image by using a trained defect detection model, wherein, at a feature extraction level of the trained defect detection model, the target detection image and the preprocessed image are subjected to feature extraction to obtain joint features; and outputting a defect detection result based on the joint features obtained at each level by using the trained defect detection model. The image problems in the target detection image are reduced by obtaining the preprocessed image corresponding to the target detection image, and the joint features are obtained by performing feature extraction on the target detection image and the preprocessed image respectively, so that the joint features can reflect the original defect condition of the target detection image and the defect condition after enhancement of the preprocessed image at the same time, thereby reducing the influence of the image problems on the defect detection and improving the detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced here. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0040] Figure 1 The flowchart of the first embodiment of the defect detection method of the present application is shown in the figure.

[0041] Figure 2 The module structure diagram of the electronic device of the present application is shown in the figure. DETAILED DESCRIPTION

[0042] It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present application.

[0043] The present application provides a defect detection method, referring to Figure 1 , Figure 1A flowchart of a first embodiment of the defect detection method of the present application is shown in the figure, and the method comprises the following steps:

[0044] In step S10, a target detection image of a target workpiece is acquired, and the target detection image is preprocessed to obtain a preprocessed image.

[0045] The target workpiece is a workpiece that needs to be detected for defects. The specific type of the target workpiece can be set based on actual needs. In this embodiment and subsequent embodiments, the target workpiece is taken as a steering knuckle casting as an example for illustration.

[0046] The target detection image is an image obtained by image acquisition of the target workpiece. The specific image acquisition method and form can be set based on actual needs. For example, the target detection image can be an RGB image, an infrared image, or an X-ray image. In this embodiment, the target detection image is taken as an X-ray image as an example for illustration.

[0047] The preprocessing is to adjust the image parameters of the target detection image to obtain a clearer preprocessed image. The specific processing method of the preprocessing can be set based on actual needs. For example, for an X-ray image, there can be problems such as low contrast and uneven gray scale distribution. Therefore, a contrast enhancement, gray scale uniformity, or other preprocessing method can be set to obtain a preprocessed image.

[0048] In step S20, the target detection image and the preprocessed image are subjected to defect detection by means of a trained defect detection model. In the feature extraction level of the trained defect detection model, the target detection image and the preprocessed image are subjected to feature extraction to obtain joint features.

[0049] In step S30, the trained defect detection model outputs a defect detection result based on the joint features obtained at each level.

[0050] The defect detection model is used to detect defects in the target detection image.

[0051] The specific type of the defect detection model can be set based on actual needs, such as a semantic segmentation model. In this embodiment, a semantic segmentation model with ConvNeXt as the backbone network is taken as the defect detection model.

[0052] In this embodiment, the target detection image and the preprocessed image are subjected to double-path encoding, and the features extracted from the target detection image and the preprocessed image are fused by means of the trained defect detection model to obtain joint features. The joint features are obtained by the joint action of the features of the target detection image and the preprocessed image. Therefore, the joint features can simultaneously reflect the defect features in the target detection image and the preprocessed image. Through the joint features, the detection capability for complex defects can be enhanced, thereby improving the detection accuracy.

[0053] The feature extraction level is a layer for extracting features of an image in the defect detection model; the specific feature extraction level can be set based on actual needs; for example, the defect detection model can include four stage modules, the first stage: through 4x4 convolution, the input image is sampled and normalized, the resolution is reduced to 1 / 4 and the channel number is expanded; the second stage: 7x7 depth separable convolution, channel normalization, 1x1 convolution expansion (MLP ratio expansion), GELU (Gaussian Error Linear Units, Gaussian Error Activation Function) activation, 1x1 convolution contraction are sequentially performed, and the input and output are fused through residual connection and dropout layer, and finally through 2x2 convolution, the resolution is reduced to 1 / 8 and the channel number is expanded; the third stage: the same operation as the second stage is performed, and finally through 2x2 convolution, the resolution is reduced to 1 / 16 and the channel number is expanded; the fourth stage: the same operation as the second stage is performed, and finally through 2x2 convolution, the resolution is reduced to 1 / 32 and the channel number is expanded; the output is a global semantic feature of 1024 channels.

[0054] As in the first convolutional layer, the target detection image and the preprocessed image are respectively extracted features and fused to obtain the corresponding joint features, and the joint features are output to the subsequent layer for convolution to further obtain the joint features corresponding to each level, until the global semantic features are finally output, and the defect detection result is obtained based on the global semantic features; the defect detection result can include whether there is a defect, defect type, size, position, image, etc.

[0055] The embodiment reduces the image problems in the target detection image by obtaining the preprocessed image corresponding to the target detection image, and extracts features from the target detection image and the preprocessed image respectively to obtain joint features, so that the joint features can reflect the original defect situation of the target detection image and the defect situation after enhancement of the preprocessed image at the same time, thereby reducing the influence of image problems on defect detection and improving detection accuracy.

[0056] Further, in the second embodiment of the defect detection method of the present application based on the first embodiment of the present application, the step S10 includes the steps of:

[0057] Step S11, obtaining the original contrast of the target detection image;

[0058] Step S12, obtaining a contrast threshold and determining whether the original contrast is less than the contrast threshold;

[0059] Step S13, if the original contrast is less than the contrast threshold, performing contrast enhancement on the target detection image to obtain the preprocessed image, wherein the contrast of the preprocessed image is greater than or equal to the contrast threshold.

[0060] If the original contrast is greater than or equal to the contrast threshold, the target detection image is not subjected to contrast enhancement.

[0061] The original contrast is an actual contrast of the target detection image.

[0062] The contrast threshold is a minimum contrast of the target detection image allowed; it can be understood that when the original contrast of the target detection image is too small, the contrast between the defects and the background in the target detection image is not obvious, and it is more difficult to identify the defects from the image, and the details of the image are lost seriously, which also affects the accuracy of defect detection. Therefore, the contrast threshold is set in the embodiment to judge the original contrast of the target detection image, and when it is detected that the original contrast is too small, the target detection image is subjected to contrast enhancement, so that the contrast of the preprocessed image obtained after the contrast enhancement can reach the contrast threshold, so that the contrast of the preprocessed image can meet the requirement of defect detection.

[0063] The specific value of the contrast threshold can be set based on actual needs.

[0064] Further, the step S10 comprises the steps of:

[0065] Step S14, obtaining an original gray variance of the target detection image.

[0066] Step S15, obtaining a gray variance threshold, and judging whether the original gray variance is greater than the gray variance threshold.

[0067] Step S16, if the original gray variance is greater than the gray variance threshold, eliminating the local gray difference of the target detection image to obtain the preprocessed image.

[0068] If the original gray variance is less than or equal to the gray variance threshold, the local gray difference of the target detection image is not eliminated.

[0069] The original gray variance is an actual gray variance of the target detection image; the gray variance can be calculated by the gray of each pixel in the target detection image.

[0070] The gray scale variance threshold is the maximum gray scale variance of the allowed target detection image. It can be understood that when the original gray scale variance of the target detection image is too large, there will be a problem of uneven gray scale distribution in the target detection image, making it difficult to identify defects from the image. Therefore, the gray scale variance threshold is set in the embodiment to judge the original gray scale variance of the target detection image. When it is detected that the original gray scale variance is too large, the local gray scale difference of the target detection image is eliminated, so that the gray scale variance of the preprocessed image obtained after elimination can be less than the gray scale variance threshold, so that the gray scale variance of the preprocessed image can meet the defect detection requirements.

[0071] The specific local gray scale difference elimination method is as follows:

[0072]

[0073] Wherein, I1 is the pixel gray scale value obtained after elimination; (x, y) is the coordinate of the pixel in the target detection image; I is the gray scale value of the pixel in the target detection image; μ is the local mean; σ is the standard deviation, and ε is a constant.

[0074] When calculating the local mean, the target detection image can be divided into small regions, and the size of the region can be determined according to the set sliding window size. The local mean is the average value of the pixel values in the small region.

[0075] The specific value of the gray scale variance threshold can be set based on actual needs.

[0076] It should be noted that the specific operation of preprocessing can include a variety, such as by judging the contrast threshold and the gray scale variance threshold of the target detection image, and determining whether contrast enhancement and local gray scale difference elimination are needed based on the specific judgment result; such as when the actual contrast of the target detection image is less than the contrast threshold and the actual gray scale variance is greater than the gray scale variance threshold, the target detection image is subjected to contrast enhancement and local gray scale difference elimination to obtain a preprocessed image.

[0077] Further, the step S12 includes:

[0078] Step S121, acquiring a first historical missed detection image, and determining a historical contrast corresponding to each of the first historical missed detection images;

[0079] Step S122, taking the maximum historical contrast as a reference contrast;

[0080] Step S123, determining the contrast threshold according to the reference contrast, wherein the contrast threshold is greater than the reference contrast.

[0081] The first historical missed detection image is an image determined by a production line after defect detection, and the image is missed due to low contrast. It can be understood that the first historical missed detection image is updated in real time based on the defect detection method of the application in the production line application. The first historical missed detection image can be determined by manual sampling inspection.

[0082] The historical contrast is the actual contrast of the first historical missed detection image.

[0083] The maximum value in the historical contrast is determined as a reference contrast. It can be understood that since the first historical missed detection image is caused by low contrast, the maximum value in the historical contrast has a high probability of being the contrast critical value that is detected and missed. Therefore, in the embodiment, the contrast threshold is determined by the reference contrast, and the contrast threshold is greater than the reference contrast, so that the contrast threshold can be greater than the contrast critical value that is detected and missed, thereby improving the optimization effect on the image with low contrast. The relative size between the contrast threshold and the reference contrast can be set based on actual needs, such as setting the contrast threshold to 1.2 times the reference contrast.

[0084] Further, the step S15 comprises:

[0085] Step S151, acquiring a second historical missed detection image, and determining a historical gray scale variance corresponding to each of the second historical missed detection images;

[0086] Step S152, taking the smallest historical gray scale variance as a reference gray scale variance;

[0087] Step S153, taking the reference gray scale variance as the gray scale variance threshold.

[0088] The second historical missed detection image is an image determined by a production line after defect detection, and the image is missed due to high gray scale variance. It can be understood that the second historical missed detection image is updated in real time based on the defect detection method of the application in the production line application. The second historical missed detection image can be determined by manual sampling inspection.

[0089] The historical gray scale variance is the actual gray scale variance of the second historical missed detection image.

[0090] The minimum value in the historical gray scale variance is determined as a reference gray scale variance. It can be understood that since the second historical missed detection image is caused by high gray scale variance, the minimum value in the historical gray scale variance has a high probability of being the gray scale variance critical value that is detected and missed. Therefore, in the embodiment, the reference gray scale variance is taken as the gray scale variance threshold, so that the gray scale variance threshold can reach the gray scale variance critical value that is detected and missed, thereby improving the optimization effect on the image with high gray scale variance.

[0091] Further, in the third embodiment of the defect detection method of the present application based on the first embodiment of the present application, the step S20 comprises the steps of:

[0092] Step S21, performing feature extraction on the target detection image to obtain original image features;

[0093] Step S22, performing feature extraction on the preprocessed image to obtain preprocessed features;

[0094] Step S23, obtaining a proportion coefficient, and performing feature fusion on the original image features and the preprocessed features based on the proportion coefficient to obtain the joint features.

[0095] The original image features are image features obtained by performing feature extraction on the target detection image.

[0096] The preprocessed features are image features obtained by performing feature extraction on the preprocessed image.

[0097] It can be understood that the original image features reflect the defect situation in the target detection image, and are more inclined to the actual real situation; the preprocessed features reflect the defect situation in the preprocessed image, and are more inclined to the clear image situation; therefore, by fusing the original image features and the preprocessed features, the joint features reflecting both the real situation and the clear situation can be obtained, and therefore the detection accuracy can be improved based on the joint features.

[0098] Specifically, the feature fusion operation can be:

[0099]

[0100] Wherein, T is the joint features; T1 is the original image features; T2 is the preprocessed features; a is the proportion coefficient.

[0101] The proportion coefficient is used to indicate the influence of the original image features on the joint features; it can be understood that the greater the proportion coefficient, the more the original image features can be highlighted, and the features of the low-contrast and uneven gray distribution image can be better retained; and the smaller the proportion coefficient, the more the preprocessed image features can be highlighted, and the features of the low-contrast and uneven gray distribution image can be better eliminated; the proportion coefficient can be set based on actual application scenarios and optimization needs, such as 50%.

[0102] Further, in the fourth embodiment of the defect detection method of the present application based on the first embodiment of the present application, the step S20 comprises the steps of:

[0103] Step S40, obtaining an initial defect detection model, wherein the defect detection model comprises an auxiliary decoding head, and a dilated spatial pyramid pooling layer is arranged in the auxiliary decoding head;

[0104] Step S50, obtaining a defect detection training sample, and training the initial defect detection model through the defect detection training sample to obtain a trained defect detection model.

[0105] The auxiliary decoding head processes the features output by the first to fourth stages of the defect detection model through an FCNHead structure; under regular sampling, local detail features will be lost, resulting in low detection accuracy of small defects by the defect detection model. Therefore, in the embodiment, an ASPP (Atrous Spatial Pyramid Pooling) is arranged in the auxiliary decoding head. The ASPP is implemented in the multi-scale context extraction stage after feature mapping, captures semantic information of different receptive fields through parallel dilated convolution, and finally outputs a segmentation prediction map matching the input resolution through 1x1 convolution and up-sampling to assist the training of the main decoding head. The dilated convolution of the ASPP can expand the receptive field without reducing the resolution, retain spatial details, and improve the detection capability for small defects.

[0106] The defect detection training sample is used to train the defect detection model; the specific acquisition method of the defect detection training sample can be set based on actual needs, for example, X-ray images of various types of knuckle castings on the production line are collected, and the defect detection training sample is obtained through artificial defect labeling.

[0107] The specific settings in the model training can be set based on actual needs, such as loss function, iteration number, etc.

[0108] After obtaining the trained defect detection model, the defect detection model can be converted into a tensorrt acceleration format and deployed to the production line for defect detection; Tensorrt is a high-performance deep learning inference optimization library that improves the inference speed of the model through optimization of graph calculation and reduction of memory occupation. The trained model can be efficiently run on GPU after being converted into an engine file by Tensorrt.

[0109] Further, in the fifth embodiment of the defect detection method of the present application based on the first embodiment of the present application, the defect detection model comprises a dropout layer, and the step S20 comprises the steps of:

[0110] Step S24, obtaining the number of real-time workpiece types and the number of dataset categories corresponding to the defect detection model;

[0111] Step S25, determining a target dropout probability corresponding to the number of real-time workpiece types according to the number of dataset categories, wherein the target dropout probability is negatively correlated with the number of real-time workpiece types;

[0112] Step S26, set the discard probability of the discard layer as the target discard probability.

[0113] The discard layer is used to randomly discard a part of neurons to prevent model overfitting.

[0114] The dataset category number is the number of categories contained in the dataset corresponding to the pre-trained model adopted by the defect detection model; for example, a pre-trained model on the ADE20K dataset is adopted, and the ADE20K dataset contains 150 categories, so the corresponding dataset category number is 150.

[0115] The real-time workpiece type number is the number of models of workpieces that need to be processed on the current production line, for example, the number of models of knuckle castings contained in the current production line; in this embodiment, the target discard probability and the real-time workpiece type number are set to be negatively correlated, when the number of models of knuckle castings is less than the dataset category number, the fewer the number of models of knuckle castings, the more the features of the dataset, at this time, the neuron discard probability is greater, and the discarded dataset features are relatively more, so the features of the knuckle casting can be highlighted more. When the number of models of knuckle castings is greater than the dataset category: at this time, the discard probability is smaller and is set to a fixed value, which can ensure that the discarded knuckle casting features are less; specifically:

[0116]

[0117] Wherein, p1 is the target discard probability, n is the real-time workpiece type number, and C1 is the dataset category number.

[0118] Taking the ADE20K dataset as an example, C1 is 150;

[0119] The number of models of knuckle castings is less than 150, and from the above formula, it can be seen that the smaller n is, the greater the probability p1 is, and the more neurons are discarded; from the category, the number of models of knuckle castings in this case is less than the number of categories of the ADE20K dataset, so more neurons are discarded, which means that more features of the dataset are discarded, and the features of the knuckle casting are discarded less, that is, the knuckle casting defect features are highlighted.

[0120] The number of models of knuckle castings is greater than 150, in this case, the probability p1 directly jumps to 0.1, that is, a fixed number of neurons are discarded; from the category, the number of models of knuckle castings in this case is greater than the number of categories of the ADE20K dataset, and discarding neurons in this case will cause more knuckle casting features to be discarded than dataset features, so setting the probability to 0.1 can preserve the knuckle casting defect features and ensure the detection accuracy.

[0121] Overall, the optimized discard layer can better extract the defect features of the knuckle casting during training, thereby improving the detection accuracy of the model.

[0122] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0123] From the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present application.

[0124] The present application also provides a defect detection device for implementing the above defect detection method, the defect detection device comprising:

[0125] A first acquisition module is configured to acquire a target detection image of a target workpiece and perform preprocessing on the target detection image to obtain a preprocessed image;

[0126] A first detection module is configured to perform defect detection on the target detection image and the preprocessed image by using a trained defect detection model, wherein feature extraction is performed on the target detection image and the preprocessed image at a feature extraction level of the trained defect detection model to obtain joint features;

[0127] A first execution module is configured to output a defect detection result based on the joint features obtained at each level by using the trained defect detection model.

[0128] The defect detection device reduces image problems in the target detection image by acquiring a preprocessed image corresponding to the target detection image, and reduces the influence of image problems on defect detection and improves detection accuracy by performing feature extraction on the target detection image and the preprocessed image respectively and obtaining joint features, so that the joint features can reflect the original defect condition of the target detection image and the defect condition after enhancement of the preprocessed image at the same time.

[0129] It should be noted that the first acquisition module in this embodiment can be configured to perform step S10 in the embodiments of the present application, the first detection module in this embodiment can be configured to perform step S20 in the embodiments of the present application, and the first execution module in this embodiment can be configured to perform step S30 in the embodiments of the present application.

[0130] Further, the first acquisition module comprises:

[0131] a first acquisition unit configured to acquire an original contrast of the target detection image;

[0132] a second acquisition unit configured to acquire a contrast threshold and determine whether the original contrast is less than the contrast threshold;

[0133] a first execution unit configured to perform contrast enhancement on the target detection image to obtain the preprocessed image if the original contrast is less than the contrast threshold, wherein the contrast of the preprocessed image is greater than or equal to the contrast threshold.

[0134] Further, the first acquisition module comprises:

[0135] a third acquisition unit configured to acquire an original gray variance of the target detection image;

[0136] a fourth acquisition unit configured to acquire a gray variance threshold and determine whether the original gray variance is greater than the gray variance threshold;

[0137] a second execution unit configured to eliminate local gray difference of the target detection image to obtain the preprocessed image if the original gray variance is greater than the gray variance threshold.

[0138] Further, the first acquisition module comprises:

[0139] a first acquisition subunit configured to acquire a first historical missed detection image and determine a historical contrast corresponding to each of the first historical missed detection images;

[0140] a first execution subunit configured to take the maximum historical contrast as a reference contrast;

[0141] a first determination subunit configured to determine the contrast threshold according to the reference contrast, wherein the contrast threshold is greater than the reference contrast.

[0142] Further, the first detection module comprises:

[0143] a first extraction unit configured to perform feature extraction on the target detection image to obtain an original image feature;

[0144] The second extraction unit is configured to perform feature extraction on the preprocessed image to obtain preprocessed features.

[0145] The fifth acquisition unit is configured to acquire a scaling coefficient, and perform feature fusion on the original image features and the preprocessed features based on the scaling coefficient to obtain the joint features.

[0146] Further, the apparatus further comprises:

[0147] The second acquisition module is configured to acquire an initial defect detection model, wherein the defect detection model comprises an auxiliary decoding head, and a dilated spatial pyramid pooling layer is arranged in the auxiliary decoding head.

[0148] The third acquisition module is configured to acquire a defect detection training sample, and train the initial defect detection model through the defect detection training sample to obtain a trained defect detection model.

[0149] Further, the defect detection model comprises a dropout layer, and the first detection module comprises:

[0150] The sixth acquisition unit is configured to acquire a real-time workpiece type quantity and a data set category quantity corresponding to the defect detection model.

[0151] The first determination unit is configured to determine a target dropout probability corresponding to the real-time workpiece type quantity according to the data set category quantity, wherein the target dropout probability is negatively correlated with the real-time workpiece type quantity.

[0152] The first setting unit is configured to set a dropout probability of the dropout layer as the target dropout probability.

[0153] Reference Figure 2 The electronic device can include a communication module 10, a memory 20, a processor 30, and the like in terms of hardware structure. In the electronic device, the processor 30 is connected with the memory 20 and the communication module 10 respectively, the memory 20 stores a computer program, the computer program is executed by the processor 30, and the computer program implements the steps of the above method embodiment when executed.

[0154] The communication module 10 can be connected with external communication devices through a network. The communication module 10 can receive a request sent by the external communication device, and can also send a request, an instruction and information to the external communication device. The external communication device can be other electronic devices, servers or Internet of Things devices, such as televisions and the like.

[0155] The memory 20 can be used to store software programs and various data. The memory 20 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as acquiring a target inspection image of the target workpiece), etc.; the data storage area may include a database, and may store data or information created based on system usage. Furthermore, the memory 20 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0156] The processor 30 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 20, and by calling data stored in the memory 20, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 30.

[0157] although Figure 2 Not shown, but the above-described electronic device may further include a circuit control module for connecting to a power supply to ensure the normal operation of other components. Those skilled in the art will understand that... Figure 2 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0158] The present invention also proposes a computer-readable storage medium having a computer program stored thereon. The computer-readable storage medium may be... Figure 2 The memory 20 in the electronic device may also be at least one of ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk, etc. The computer-readable storage medium includes a number of instructions to cause a terminal device with a processor (which may be a television, automobile, mobile phone, computer, server, terminal, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0159] In the present application, the terms "first", "second", "third", "fourth", "fifth" are only for the purpose of description, and cannot be understood as indicating or implying relative importance, and for those skilled in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0160] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms is not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0161] Although the embodiments of the present application have been shown and described above, the scope of protection of the present application is not limited thereto, and it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications and replacements to the above embodiments within the scope of the present application, and these changes, modifications and replacements shall be covered within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims.

Claims

1. A defect detection method characterized by, The defect detection method comprises: obtaining a target detection image of a target workpiece, and pre-processing the target detection image to obtain a pre-processed image, wherein the pre-processing comprises adjusting image parameters of the target detection image; performing defect detection on the target detection image and the pre-processed image by using a trained defect detection model, wherein feature extraction is performed on the target detection image and the pre-processed image at a feature extraction level of the trained defect detection model to obtain joint features; outputting a defect detection result based on the joint features obtained at each level by using the trained defect detection model; the pre-processing of the target detection image to obtain a pre-processed image comprises: obtaining an original contrast of the target detection image; obtaining a contrast threshold, and determining whether the original contrast is less than the contrast threshold; if the original contrast is less than the contrast threshold, performing contrast enhancement on the target detection image to obtain the pre-processed image, wherein the contrast of the pre-processed image is greater than or equal to the contrast threshold; the obtaining of the contrast threshold comprises: obtaining a first historical missed detection image, and determining a historical contrast corresponding to each first historical missed detection image; taking the maximum historical contrast as a reference contrast; determining the contrast threshold based on the reference contrast, wherein the contrast threshold is greater than the reference contrast.

2. The defect detection method of claim 1, wherein, the pre-processing of the target detection image to obtain a pre-processed image comprises: obtaining an original gray variance of the target detection image; obtaining a gray variance threshold, and determining whether the original gray variance is greater than the gray variance threshold; if the original gray variance is greater than the gray variance threshold, eliminating local gray differences of the target detection image to obtain the pre-processed image.

3. The defect detection method of claim 1, wherein, the feature extraction of the target detection image and the pre-processed image to obtain joint features comprises: performing feature extraction on the target detection image to obtain original image features; performing feature extraction on the pre-processed image to obtain pre-processed features; obtaining a proportionality coefficient, and performing feature fusion on the original image features and the pre-processed features based on the proportionality coefficient to obtain the joint features.

4. The defect detection method of claim 1, wherein, the defect detection on the target detection image and the pre-processed image by using the trained defect detection model comprises: obtaining an initial defect detection model, wherein the defect detection model comprises an auxiliary decoding head, and a hole spatial pyramid pooling layer is arranged in the auxiliary decoding head; obtaining a defect detection training sample, and training the initial defect detection model by using the defect detection training sample to obtain a trained defect detection model.

5. The defect detection method of claim 1, wherein, the defect detection model comprises a dropout layer, and the defect detection on the target detection image and the pre-processed image by using the trained defect detection model comprises: obtaining a number of real-time workpiece types and a number of dataset categories corresponding to the defect detection model; determine a target discard probability corresponding to the number of real-time workpiece types according to the number of dataset categories, wherein the target discard probability is negatively correlated with the number of real-time workpiece types; set a discard probability of the discard layer as the target discard probability.

6. A defect detection apparatus characterized by comprising: The defect detection device comprises: The first acquisition module is configured to acquire a target detection image of a target workpiece, and to pre-process the target detection image to obtain a pre-processed image, wherein the pre-processing comprises adjusting image parameters of the target detection image; The first detection module is configured to perform defect detection on the target detection image and the pre-processed image by using a trained defect detection model, wherein the trained defect detection model is configured to extract features of the target detection image and the pre-processed image to obtain joint features at a feature extraction level of the trained defect detection model; The first execution module is configured to output a defect detection result based on the joint features obtained at each level by using the trained defect detection model. The first acquisition module comprises: The first acquisition unit is configured to acquire an original contrast of the target detection image; The second acquisition unit is configured to acquire a contrast threshold, and to determine whether the original contrast is less than the contrast threshold; The first execution unit is configured to perform contrast enhancement on the target detection image to obtain the pre-processed image if the original contrast is less than the contrast threshold, wherein a contrast of the pre-processed image is greater than or equal to the contrast threshold; The first acquisition module comprises: The third acquisition unit is configured to acquire an original gray variance of the target detection image; The fourth acquisition unit is configured to acquire a gray variance threshold, and to determine whether the original gray variance is greater than the gray variance threshold; The second execution unit is configured to eliminate local gray differences of the target detection image to obtain the pre-processed image if the original gray variance is greater than the gray variance threshold.

7. An electronic device, comprising: The electronic device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is executed by the processor to implement the steps of the defect detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to implement the steps of the defect detection method according to any one of claims 1 to 5.

Images