Defect detection method, device and equipment and storage medium

Abstract

The invention provides a defect detection method, apparatus and device, and a storage medium. The method comprises the steps of obtaining a to-be-detected product image and detection demand information; determining a target model adjustment parameter according to the detection demand information; determining a target defect detection model based on the target model adjustment parameters and a standard defect detection model; the standard defect detection model is obtained by training a to-be-trained model in advance based on first sample defect information, second sample defect information, a first label of the first sample defect information and a second label of the second sample defect information, and the first sample defect information is information representing existence of defects; the second sample defect information is information representing that defects do not exist; and inputting the to-be-detected product image into the target defect detection model to obtain a defect detection result. By adopting the method, the production efficiency is improved while the product quality is controlled.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a defect detection method, apparatus, device and storage medium. Background Art

[0002] Modern industrial production places stringent demands on product appearance quality. Therefore, detecting surface defects is crucial for ensuring this quality. Surface defects range from in-specification defects that do not affect basic product functionality to out-of-specification defects that severely impact product quality. To improve production efficiency while controlling product quality, detecting both in-specification and out-of-specification defects has become a pressing technical challenge. Summary of the Invention

[0003] The present application provides a defect detection method, apparatus, device and storage medium to at least solve the above technical problems existing in the prior art.

[0004] In a first aspect of the present application, a defect detection method is provided, the method comprising:

[0005] Obtain images of products to be inspected and inspection requirement information;

[0006] Determining target model adjustment parameters according to the detection requirement information;

[0007] Determining a target defect detection model based on the target model adjustment parameters and a standard defect detection model; the standard defect detection model is obtained by pre-training a to-be-trained model based on first sample defect information, second sample defect information, a first label of the first sample defect information, and a second label of the second sample defect information, wherein the first sample defect information is information indicating the presence of a defect, and the second sample defect information is information indicating the absence of a defect;

[0008] The image of the product to be inspected is input into the target defect detection model to obtain a defect detection result.

[0009] In one embodiment, the training process of the standard defect detection model includes:

[0010] Obtain multiple sample product images;

[0011] determining, based on the plurality of sample product images, a plurality of first sample defect information and a plurality of second sample defect information, a first label for the first sample defect information and a second label for the second sample defect information;

[0012] resampling the plurality of first sample defect information and the plurality of second sample defect information based on the model adjustment parameter to obtain a sample defect set consisting of the plurality of sampled sample defect information;

[0013] Input the sample defect information of the sample defect set into the to-be-trained model to obtain the classification result corresponding to the sample defect information;

[0014] Determining a loss function value of the to-be-trained model based on the classification result and the label of the sample defect information;

[0015] If the loss function value is less than or equal to the first preset loss threshold or the current iteration reaches the first preset number of iterations, the model to be trained is determined as the standard defect detection model; otherwise, new sample defect information in the sample defect set is obtained, and the step of inputting the sample defect information of the sample defect set into the model to be trained is returned to execute, and the standard defect detection model corresponds to the model adjustment parameters.

[0016] In one embodiment, determining, based on the plurality of sample product images, a plurality of first sample defect information and a plurality of second sample defect information, a first label for the first sample defect information, and a second label for the second sample defect information includes:

[0017] For each sample product image, identifying defect information included in the sample product image, each sample product image including at least one piece of defect information;

[0018] Detect whether the defect represented by each defect information exists on the corresponding product;

[0019] If so, determining the defect information as first sample defect information, and marking the first sample defect information with a first label indicating the presence of the defect;

[0020] If not, the defect information is determined as second sample defect information, and a second label indicating that the defect does not exist is marked on the second sample defect information.

[0021] In one embodiment, distribution of each piece of first sample defect information and each piece of second sample defect information included in the sampled defect set satisfies a Gaussian distribution.

[0022] In one embodiment, the model to be trained includes at least two classifiers;

[0023] The resampling of the plurality of first sample defect information and the plurality of second sample defect information based on the model adjustment parameter to obtain a sample defect set consisting of the plurality of sampled sample defect information includes:

[0024] resampling the plurality of first sample defect information and the plurality of second sample defect information at least twice based on the model adjustment parameters, wherein the plurality of sample defect information obtained after each resampling constitutes a sample defect set, and each sample defect set corresponds to a classifier;

[0025] The step of inputting the sample defect information of the sample defect set into the to-be-trained model to obtain the classification result corresponding to the sample defect information includes:

[0026] For each sampled defect set, the sample defect information of the sampled defect set is input into the model to be trained to obtain the classification result corresponding to the sample defect information.

[0027] In one embodiment, determining a target defect detection model based on the model adjustment parameters and a standard defect detection model includes:

[0028] The standard defect detection model is updated based on the target model adjustment parameters to obtain a target defect detection model.

[0029] In one embodiment, the updating of the standard defect detection model based on the target model adjustment parameters to obtain the target defect detection model includes:

[0030] resampling the plurality of first sample defect information and the plurality of second sample defect information at least twice based on the target model adjustment parameters, wherein the plurality of sample defect information obtained after each resampling constitutes a sample defect set, and each sample defect set corresponds to a classifier;

[0031] For each sampled defect set, input the sample defect information of the sampled defect set into the standard defect detection model to obtain the detection results corresponding to the sample defect information;

[0032] Determining a loss function value of the standard defect detection model based on the detection result and the label of the sample defect information;

[0033] If the loss function value is less than or equal to the second preset loss threshold or the current iteration reaches the second preset number of iterations, the standard defect detection model is determined as the target defect detection model; otherwise, new sample defect information in the sample defect set is obtained, and the step of inputting the sample defect information of the sample defect set into the standard defect detection model is returned to execute.

[0034] In one embodiment, determining a target defect detection model based on the model adjustment parameters and a standard defect detection model includes:

[0035] Based on the corresponding relationship between each parameter and each standard defect detection model, the standard defect detection model corresponding to the target model adjustment parameter is determined as the target defect detection model.

[0036] In one embodiment, the detection requirement information and the target model adjustment parameters satisfy:

[0037] The lower the detection rate requirement for defects within the product specifications represented by the detection requirement information, the larger the target model adjustment parameter.

[0038] In a second aspect of the present application, a defect detection device is provided, comprising:

[0039] Information acquisition module, used to obtain images of products to be inspected and inspection requirement information;

[0040] A parameter determination module, configured to determine target model adjustment parameters according to the detection requirement information;

[0041] a model determination module, configured to determine a target defect detection model based on the target model adjustment parameters and a standard defect detection model; the standard defect detection model is obtained by pre-training a to-be-trained model based on first sample defect information, second sample defect information, a first label of the first sample defect information, and a second label of the second sample defect information, wherein the first sample defect information is information indicating the presence of a defect, and the second sample defect information is information indicating the absence of a defect;

[0042] The defect detection module is used to input the image of the product to be detected into the target defect detection model to obtain a defect detection result.

[0043] In one embodiment, the apparatus further includes a model training module for performing a training process of the standard defect detection model:

[0044] Acquire multiple sample product images; determine multiple first sample defect information and multiple second sample defect information, first labels of the first sample defect information and second labels of the second sample defect information based on the multiple sample product images; resample the multiple first sample defect information and the multiple second sample defect information based on the model adjustment parameters to obtain a sample defect set composed of multiple sampled sample defect information; input the sample defect information of the sample defect set into the model to be trained to obtain a classification result corresponding to the sample defect information; determine the loss function value of the model to be trained based on the classification result and the label of the sample defect information; if the loss function value is less than or equal to a first preset loss threshold or the current iteration reaches a first preset number of iterations, determine the model to be trained as a standard defect detection model, otherwise, obtain new sample defect information in the sample defect set, return to execute the step of inputting the sample defect information of the sample defect set into the model to be trained, and the standard defect detection model corresponds to the model adjustment parameters.

[0045] In one possible implementation, the model training module is specifically used to identify the defect information included in each sample product image, each sample product image including at least one defect information; detect whether the defect represented by each defect information exists on the corresponding product; if so, determine the defect information as first sample defect information, and annotate the first sample defect information with a first label representing the existence of the defect; if not, determine the defect information as second sample defect information, and annotate the second sample defect information with a second label representing the absence of the defect.

[0046] In one embodiment, distribution of each piece of first sample defect information and each piece of second sample defect information included in the sampled defect set satisfies a Gaussian distribution.

[0047] In one embodiment, the model to be trained includes at least two classifiers;

[0048] The model training module is specifically configured to resample the plurality of first sample defect information and the plurality of second sample defect information at least twice based on the model adjustment parameters, wherein the plurality of sample defect information obtained after each resampling constitutes a sample defect set, and each sample defect set corresponds to a classifier;

[0049] The model training module is specifically configured to input the sample defect information of each sample defect set into the model to be trained, and obtain a classification result corresponding to the sample defect information.

[0050] In one possible implementation manner, the model determination module is specifically configured to update the standard defect detection model based on the target model adjustment parameters to obtain a target defect detection model.

[0051] In one possible implementation manner, the model determination module is specifically used to resample multiple first sample defect information and multiple second sample defect information at least twice based on the target model adjustment parameters, and the multiple sample defect information obtained after each resampling constitute a sampling defect set, and each sampling defect set corresponds to a classifier; for each sampling defect set, the sample defect information of the sampling defect set is input into the standard defect detection model to obtain the detection result corresponding to the sample defect information; based on the detection result and the label of the sample defect information, the loss function value of the standard defect detection model is determined; if the loss function value is less than or equal to the second preset loss threshold or the current iteration reaches the second preset number of iterations, the standard defect detection model is determined as the target defect detection model, otherwise, new sample defect information in the sampling defect set is obtained, and the step of inputting the sample defect information of the sampling defect set into the standard defect detection model is returned.

[0052] In one possible implementation manner, the model determination module is specifically configured to determine the standard defect detection model corresponding to the target model adjustment parameter as the target defect detection model based on the correspondence between each parameter and each standard defect detection model.

[0053] In one possible implementation manner, the detection requirement information and the target model adjustment parameter satisfy the following conditions: the lower the detection rate requirement for defects within the product specifications represented by the detection requirement information, the larger the target model adjustment parameter.

[0054] According to a third aspect of the present application, an electronic device is provided, including:

[0055] at least one processor; and

[0056] a memory communicatively connected to the at least one processor; wherein,

[0057] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0058] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.

[0059] The defect detection method, apparatus, equipment and storage medium of the present application obtain an image of a product to be inspected and detection requirement information; determine target model adjustment parameters based on the detection requirement information; determine a target defect detection model based on the target model adjustment parameters and a standard defect detection model; the standard defect detection model is obtained by pre-training a to-be-trained model based on first sample defect information, second sample defect information, a first label of the first sample defect information, and a second label of the second sample defect information, wherein the first sample defect information is information characterizing the presence of a defect, and the second sample defect information is information characterizing the absence of a defect; the image of the product to be inspected is input into the target defect detection model to obtain a defect detection result. In this application, the corresponding target model adjustment parameters can be determined by the product's detection requirement information. Different detection requirements with high and low requirements for defects within product specifications can correspond to different model adjustment parameters. The target model adjustment parameters are used to adjust the standard defect detection model to obtain a target defect detection model. Alternatively, a standard defect detection model corresponding to the target model adjustment parameters is obtained from multiple standard defect detection models as a target defect detection model, so that the obtained target defect detection model can detect defects according to the needs. For requirements with low requirements for defects within product specifications, the target defect detection model can be used to improve the missed detection rate and filter out some defects within the specifications, thereby achieving the goal of improving production efficiency while controlling product quality.

[0060] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an illustrative and non-limiting manner, in which:

[0062] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0063] Figure 1 A schematic diagram of a process of a defect detection method provided by an embodiment of the present application is shown;

[0064] Figure 2 A schematic diagram of a training process of a standard defect detection model provided in an embodiment of the present application is shown;

[0065] Figure 3 A schematic diagram of a target defect detection model determination process provided by an embodiment of the present application is shown;

[0066] Figure 4 A schematic diagram showing the variation of the number of defect detections with model adjustment parameters provided by an embodiment of the present application is shown;

[0067] Figure 5 A schematic structural diagram of a defect detection device provided in an embodiment of the present application is shown;

[0068] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0069] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0070] Since in-specification defects do not affect product functionality, in order to control product quality while improving production efficiency, this application provides a defect detection method, apparatus, device, and storage medium for detecting both in-specification and out-of-specification defects. The defect detection method provided in this application can be applied to any electronic device capable of image processing, including but not limited to computers, mobile phones, and tablets.

[0071] The technical solutions of the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0072] Figure 1 A schematic diagram of a defect detection method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0073] S101, obtaining an image of a product to be inspected and inspection requirement information.

[0074] In this application, the products to be inspected may include electronic products and mechanical equipment, such as mobile phones, mobile phone components, computers, tablet computers, and machine tools. The images of the products to be inspected may be images of the product's exterior or the surface of its internal components. In this application, an image acquisition device may be used to capture images of the product's exterior. For example, the image acquisition device may be a high-definition camera or a high-resolution mobile phone camera.

[0075] In this application, testing requirement information includes a user's defect detection requirements for the product to be tested. For example, testing requirement information may include the detection rate of within-specification defects for the product to be tested. Within-specification defects refer to defects in the product's appearance that do not affect its functionality. For example, color differences or stains on the product's appearance that are not noticeable to the naked eye are all considered within-specification defects.

[0076] S102: Determine target model adjustment parameters according to detection requirement information.

[0077] In this application, the value of the target model adjustment parameter is greater than or equal to 0 and less than or equal to 1.

[0078] S103 , determining a target defect detection model based on the target model adjustment parameters and the standard defect detection model.

[0079] The standard defect detection model is obtained by pre-training the model to be trained based on the first sample defect information, the second sample defect information, the first label of the first sample defect information, and the second label of the second sample defect information. The first sample defect information is information representing the existence of a defect, and the second sample defect information is information representing the non-existence of a defect.

[0080] In this application, the first sample defect information may include information indicating that the product defect is an out-of-specification defect. An out-of-specification defect refers to a defect in the product that affects product quality. For example, an out-of-specification defect may include: a large scratch, a large stain, or damage on the product surface. The first label of the first sample defect information is a label used to indicate that the product defect is an out-of-specification defect.

[0081] The second sample defect information may include information indicating that the product defect is an in-specification defect or information indicating that the product does not have a defect. The second label of the second sample defect information is a label used to indicate that the product defect is an out-of-specification defect or that the product does not have a defect.

[0082] In this application, a standard defect detection model can be used to detect product defects.

[0083] S104: Input the image of the product to be inspected into the target defect detection model to obtain a defect detection result.

[0084] In this application, the defect detection result may include a result indicating whether the product to be inspected has a defect or not. The defect detection result may also include a result indicating the type of defect present in the product to be inspected. For example, if the product to be inspected is a mobile phone, the defect detection result may include a result indicating that the mobile phone has a defect and the defect type is a scratch defect on the exterior.

[0085] The defect detection method provided in this application is used to obtain an image of a product to be inspected and detection requirement information; the target model adjustment parameters are determined according to the detection requirement information; the target defect detection model is determined based on the target model adjustment parameters and the standard defect detection model; the standard defect detection model is obtained by pre-training a to-be-trained model based on first sample defect information, second sample defect information, a first label of the first sample defect information, and a second label of the second sample defect information, the first sample defect information is information characterizing the presence of a defect, and the second sample defect information is information characterizing the absence of a defect; the image of the product to be inspected is input into the target defect detection model to obtain a defect detection result. In this application, the corresponding target model adjustment parameters can be determined by the product's detection requirement information. Different detection requirements with high and low requirements for defects within product specifications can correspond to different model adjustment parameters. The target model adjustment parameters are used to adjust the standard defect detection model to obtain a target defect detection model. Alternatively, a standard defect detection model corresponding to the target model adjustment parameters is obtained from multiple standard defect detection models as a target defect detection model, so that the obtained target defect detection model can adapt to the needs to detect defects. For requirements with low requirements for defects within product specifications, the target defect detection model can be used to increase the missed detection rate and filter out some defects within the specifications, thereby achieving the goal of improving production efficiency while controlling product quality.

[0086] In one possible embodiment, the lower the detection rate requirement for defects within the product specifications represented by the detection requirement information, the larger the target model adjustment parameter. For example, the detection requirement information may include the detection rate for defects within the product specifications for the product to be detected. When the detection rate for defects within the product specifications for the product to be detected is higher, the higher the detection rate requirement for defects within the product specifications represented by the detection requirement information; when the detection rate for defects within the product specifications for the product to be detected is lower, the lower the detection rate requirement for defects within the product specifications represented by the detection requirement information. The lower the detection rate requirement for defects within the product specifications represented by the detection requirement information, the more relaxed the requirements for defects that are invisible to the naked eye on the appearance of the product to be detected and do not affect the product's functional use. The target model adjustment parameter can be set to a larger value, so that the target defect detection model determined by the target model adjustment parameter becomes relaxed in detecting defects within the product specifications.

[0087] In one possible implementation, Figure 2 A schematic diagram of a training process of a standard defect detection model provided in an embodiment of the present application is shown. Figure 2 As shown in Figure 1, the training process of the standard defect detection model includes:

[0088] S201, obtaining multiple sample product images.

[0089] In this application, a sample product image refers to an image of a sample product with defects in its appearance or surface. Sample products may include mobile phones, computers, and electronic components. In this application, an image acquisition device may be used to capture multiple images of the appearance of sample products. A defect recognition algorithm is used to identify defects in each captured image, and images containing defects are used as sample product images. Defects may include scratches, stains, voids, and impurities. The defect recognition algorithm may employ edge detection or image segmentation algorithms, for example.

[0090] S202 : Determine, based on a plurality of sample product images, a plurality of first sample defect information and a plurality of second sample defect information, a first label for the first sample defect information, and a second label for the second sample defect information.

[0091] In this application, the first sample defect information may include information indicating that the sample product's defect is an out-of-specification defect. The first label is a label used to indicate that the sample product's defect is an out-of-specification defect. The second sample defect information may include information indicating that the sample product's defect is an in-specification defect or information indicating that the sample product does not have a defect. The second label is a label used to indicate that the product's defect is an out-of-specification defect or that the product does not have a defect.

[0092] In one possible implementation, determining, based on multiple sample product images, multiple first sample defect information and multiple second sample defect information, first labels for the first sample defect information and second labels for the second sample defect information may include steps A1-A4:

[0093] Step A1: for each sample product image, identify defect information included in the sample product image, where each sample product image includes at least one defect information.

[0094] Step A2: Detect whether the defect represented by each defect information exists on the corresponding product.

[0095] Step A3: If the defect exists, the defect information is determined as first sample defect information, and a first label representing the existence of the defect is marked on the first sample defect information.

[0096] Step A4: If the defect does not exist, the defect information is determined as second sample defect information, and a second label indicating that the defect does not exist is marked on the second sample defect information.

[0097] In the present application, the sample product image may contain defect information of at least one defect. Each defect information can be manually inspected. If the defect information indicates that the defect of the sample product is an out-of-specification defect, the defect information is determined as first sample defect information and a first label is annotated for the first sample defect information. If the defect information indicates that the defect of the sample product is an in-specification defect or the defect indicated by the defect information does not exist, the defect information is determined as second sample defect information and a second label is annotated for the second sample defect information. In the present application, a label annotation model trained using a deep learning model can also be used to classify and annotate sample product images for defects.

[0098] In the present application, the first label and the second label may be in the form of numbers, character strings, or a combination of numbers and character strings.

[0099] S203 : resample the plurality of first sample defect information and the plurality of second sample defect information based on the model adjustment parameter to obtain a sample defect set consisting of the plurality of sampled sample defect information.

[0100] In the present application, the distribution of each first sample defect information and each second sample defect information included in the sample defect set satisfies a Gaussian distribution. In the present application, a Gaussian kernel can be used to resample the multiple first sample defect information and multiple second sample defect information based on a model adjustment parameter. The model adjustment parameter can be the standard deviation parameter α of the Gaussian kernel. The model adjustment parameter can adjust the balance between the ratio of the first sample defect information and the second sample defect information in the resampling process and the total sample defect information. A larger value of the model adjustment parameter can make the Gaussian kernel smoother, which can increase the proportion of the second sample defect information in the resampled sample defect set, thereby reducing the detection rate of the trained standard defect detection model for within-specification defects. A smaller value of the model adjustment parameter can make the Gaussian kernel sharper, which can reduce the proportion of the second sample defect information in the resampled sample defect set, thereby improving the detection rate of the trained standard defect detection model for within-specification defects.

[0101] S204: Input the sample defect information of the sample defect set into the model to be trained to obtain a classification result corresponding to the sample defect information.

[0102] In the present application, the sampled defect set may include a plurality of first sample defect information and a first label for each first sample defect information, and a plurality of second sample defect information and a second label corresponding to each second sample defect information.

[0103] S205 , determining a loss function value of the model to be trained based on the classification result and the label of the sample defect information.

[0104] In this application, a logarithmic loss function or an exponential loss function can be used to determine the loss function value of the model to be trained.

[0105] S206: If the loss function value is less than or equal to the first preset loss threshold or the current iteration reaches the first preset number of iterations, the model to be trained is determined to be the standard defect detection model; otherwise, new sample defect information in the sample defect set is obtained, and the step of inputting the sample defect information of the sample defect set into the model to be trained is returned to execute, and the standard defect detection model corresponds to the model adjustment parameters.

[0106] In the present application, the first preset loss threshold can be set according to application requirements, for example, the first preset loss threshold can be set to 0.1 or 0.15, etc. The first preset number of iterations can be set to 100 or 200, etc.

[0107] In this application, multiple different model adjustment parameters can be set. For each model adjustment parameter, multiple first sample defect information and multiple second sample defect information can be resampled based on the model adjustment parameter to obtain a corresponding sampled defect set. The sampled defect set is used to train the to-be-trained model to obtain a standard defect detection model, which corresponds to the model adjustment parameter.

[0108] In this application, the value of the model adjustment parameter is greater than or equal to 0 and less than or equal to 1. The larger the value of the model adjustment parameter, the lower the detection rate of the corresponding standard defect detection model for within-specification defects. For use scenarios with lower requirements for within-specification defects, this standard defect detection model can improve product production efficiency while ensuring defect detection accuracy.

[0109] In one possible implementation, the model to be trained may include at least two classifiers. Resampling multiple first sample defect information and multiple second sample defect information based on model adjustment parameters to obtain a sample defect set consisting of multiple sample defect information includes: resampling the multiple first sample defect information and multiple second sample defect information at least twice based on the model adjustment parameters, wherein the multiple sample defect information obtained after each resampling constitutes a sample defect set, and each sample defect set corresponds to a classifier. When the model to be trained includes at least two classifiers, inputting the sample defect information of the sample defect set into the model to be trained to obtain a classification result corresponding to the sample defect information includes: for each sample defect set, inputting the sample defect information of the sample defect set into the model to be trained to obtain a classification result corresponding to the sample defect information.

[0110] In this application, the model to be trained can adopt a random forest model, and the random forest model can include two or more decision trees as classifiers.

[0111] Each sample defect set used to train each classifier in the same random forest model is obtained by resampling based on the same model adjustment parameters. In the present application, for each classifier in the same random forest model, multiple first sample defect information and multiple second sample defect information can be resampled based on the model adjustment parameters corresponding to the model to obtain a sample defect set consisting of multiple sampled sample defect information. The sample defect set includes multiple first sample defect information and a first label for each first sample defect information, and multiple second sample defect information and a second label for each second sample defect information. The classifier can be trained using the sample defect set. Specifically, for the sample defect set, the sample defect information of the sample defect set can be input into the model to be trained to obtain the classification result corresponding to the sample defect information; based on the classification result and the label of the sample defect information, the loss function value of the model to be trained is determined; if the loss function value is less than or equal to the first preset loss threshold or the current iteration reaches the first preset number of iterations, the model to be trained is determined as the standard defect detection model; otherwise, new sample defect information in the sample defect set is obtained, and the process of inputting the sample defect information of the sample defect set into the model to be trained is returned to.

[0112] In this application, corresponding to each classifier in the same random forest model, multiple first sample defect information and multiple second sample defect information can be resampled multiple times based on the model adjustment parameters corresponding to the model. The multiple sample defect information obtained after each resampling constitute a sampling defect set. Each sampling defect set corresponds to a classifier, and the sampling defect set obtained after each resampling is different.

[0113] In this application, the sample defect information is resampled multiple times to obtain multiple different sampled defect sets for multiple classifiers using a random forest model. Using multiple classifiers for defect detection can reduce the defect detection bias of the model, thereby improving the accuracy of the model for defect detection.

[0114] In one possible implementation, determining the target defect detection model based on the model adjustment parameters and the standard defect detection model may include: updating the standard defect detection model based on the target model adjustment parameters to obtain the target defect detection model. Specifically, Figure 3 FIG. 1 shows a flow chart of a target defect detection model determination process provided by an embodiment of the present application. Figure 3 As shown, the standard defect detection model is updated based on the target model adjustment parameters to obtain the target defect detection model, including:

[0115] S301 , resampling multiple first sample defect information and multiple second sample defect information at least twice based on target model adjustment parameters, wherein the multiple sample defect information obtained after each resampling constitute a sample defect set, and each sample defect set corresponds to a classifier.

[0116] In this application, the relationship between the detection requirement information and the target model adjustment parameters is that the lower the detection rate requirement for defects within the product specifications represented by the detection requirement information, the larger the target model adjustment parameter. The larger the target model adjustment parameter, the higher the proportion of second sample defect information representing defects within the specifications or the absence of defects in the resampled sample defect set.

[0117] In this application, the plurality of first sample defect information and the plurality of second sample defect information may be resampled two or more times, and the sample defect sets obtained after each resampling are different. The standard defect detection model may include two or more classifiers, one classifier for each sample defect set.

[0118] S302 : For each sampled defect set, input the sample defect information of the sampled defect set into a standard defect detection model to obtain a detection result corresponding to the sample defect information.

[0119] In this application, for each sampled defect set, the sample defect information of the sampled defect set can be input into a standard defect detection model to obtain the corresponding detection results of the standard defect detection model. The detection results can be either out-of-specification defects or in-specification defects. Based on the detection results and the labels of the sample defect information, the loss function value of the standard defect detection model is determined.

[0120] S303: Determine the loss function value of the standard defect detection model based on the detection results and the labels of the sample defect information.

[0121] In this application, a logarithmic loss function or an exponential loss function can be used to determine the loss function value of the model to be trained.

[0122] S304: If the loss function value is less than or equal to the second preset loss threshold or the current iteration reaches the second preset number of iterations, the standard defect detection model is determined as the target defect detection model; otherwise, new sample defect information in the sample defect set is obtained, and the step of inputting the sample defect information of the sample defect set into the standard defect detection model is returned.

[0123] In the present application, the second preset loss threshold can be set according to application requirements, for example, the second preset loss threshold can be set to 0.08 or 0.1, etc. The second preset number of iterations can be set to 100 or 200, etc.

[0124] In another possible implementation, the target defect detection model is determined based on the model adjustment parameters and the standard defect detection model, including: based on the correspondence between each parameter and each standard defect detection model, determining the standard defect detection model corresponding to the target model adjustment parameter as the target defect detection model.

[0125] In this application, multiple different model adjustment parameters can be pre-set. For each model adjustment parameter, multiple first sample defect information and multiple second sample defect information are resampled to obtain a sampled defect set. The sampled defect set is used to train the to-be-trained model to obtain a standard defect detection model, which is then used as the standard defect detection model corresponding to the model adjustment parameter. This allows for the correspondence between multiple model adjustment parameters and the standard defect detection model.

[0126] After the target model adjustment parameters are obtained, the standard defect detection model corresponding to the target model adjustment parameters can be used as the target defect detection model.

[0127] In this application, the value of the model adjustment parameter is greater than or equal to 0 and less than or equal to 1. Figure 4 A schematic diagram showing the variation of the number of defect detections with model adjustment parameters provided in an embodiment of the present application is shown. Figure 4 The vertical axis represents the size of the model adjustment parameter, and the value of the model adjustment parameter is between 0 and 1 (including 0 and 1). The horizontal axis represents the number of over-detection and missed detection. Figure 4 As shown in Figure 1, the 1342 sample defect information includes 869 in-specification defect information and 473 out-of-specification defect information. The standard defect detection model corresponding to the model adjustment parameters with different values ​​is used to detect the 1342 sample defect information respectively, as shown in Figure 1. Figure 4 As shown, larger values ​​for the model adjustment parameters increase the missed detection rate and decrease the number of passed inspections for sample defect information. Smaller values ​​for the model adjustment parameters also decrease the missed detection rate and increase the number of passed inspections for sample defect information. In other words, larger values ​​for the model adjustment parameters increase the missed detection rate and decrease the number of passed inspections for defect information, indicating that the standard defect detection model corresponding to the model adjustment parameters has a lower detection rate for within-specification defects. For use cases with lower requirements for within-specification defects, this standard defect detection model can improve product production efficiency while ensuring defect detection accuracy.

[0128] In this application, the defect detection capability of the defect detection model can be flexibly adjusted by adjusting the model parameters, so that in actual applications, the detection strategy can be dynamically adjusted according to the detection requirements at different stages of the product production cycle, and the defect detection rate can be accurately controlled to ensure a dual improvement in production efficiency and product quality.

[0129] Based on the same inventive concept, according to the defect detection method provided in the above embodiment of the present application, another embodiment of the present application also provides a defect detection device, the structural diagram of which is shown in FIG. Figure 5 As shown, specifically including:

[0130] Information acquisition module 501, used to obtain images of products to be inspected and inspection requirement information;

[0131] Parameter determination module 502, used to determine target model adjustment parameters according to detection requirement information;

[0132] A model determination module 503 is configured to determine a target defect detection model based on target model adjustment parameters and a standard defect detection model; the standard defect detection model is obtained by pre-training a to-be-trained model based on first sample defect information, second sample defect information, a first label of the first sample defect information, and a second label of the second sample defect information, wherein the first sample defect information is information indicating the presence of a defect, and the second sample defect information is information indicating the absence of a defect;

[0133] The defect detection module 504 is used to input the image of the product to be detected into the target defect detection model to obtain the defect detection result.

[0134] The defect detection device provided by the present application is used to obtain an image of a product to be inspected and detection requirement information; the target model adjustment parameters are determined according to the detection requirement information; the target defect detection model is determined based on the target model adjustment parameters and the standard defect detection model; the standard defect detection model is obtained by pre-training a to-be-trained model based on first sample defect information, second sample defect information, a first label of the first sample defect information, and a second label of the second sample defect information, the first sample defect information is information characterizing the presence of a defect, and the second sample defect information is information characterizing the absence of a defect; the image of the product to be inspected is input into the target defect detection model to obtain a defect detection result. In this application, the corresponding target model adjustment parameters can be determined by the product's detection requirement information. Different detection requirements with high and low requirements for defects within product specifications can correspond to different model adjustment parameters. The target model adjustment parameters are used to adjust the standard defect detection model to obtain a target defect detection model. Alternatively, a standard defect detection model corresponding to the target model adjustment parameters is obtained from multiple standard defect detection models as a target defect detection model, so that the obtained target defect detection model can adapt to the needs to detect defects. For requirements with low requirements for defects within product specifications, the target defect detection model can be used to increase the missed detection rate and filter out some defects within the specifications, thereby achieving the goal of improving production efficiency while controlling product quality.

[0135] In one embodiment, the apparatus further includes a model training module (not shown in the figure) for performing a training process of a standard defect detection model:

[0136] Acquire multiple sample product images; determine multiple first sample defect information and multiple second sample defect information, first labels of the first sample defect information and second labels of the second sample defect information based on the multiple sample product images; resample the multiple first sample defect information and the multiple second sample defect information based on the model adjustment parameters to obtain a sample defect set consisting of multiple sampled sample defect information; input the sample defect information of the sample defect set into the model to be trained to obtain the classification results corresponding to the sample defect information; determine the loss function value of the model to be trained based on the classification results and the labels of the sample defect information; if the loss function value is less than or equal to a first preset loss threshold or the current iteration reaches a first preset number of iterations, determine the model to be trained as a standard defect detection model, otherwise, obtain new sample defect information in the sample defect set, return to the step of inputting the sample defect information of the sample defect set into the model to be trained, and the standard defect detection model corresponds to the model adjustment parameters.

[0137] In one embodiment, the model training module is specifically used to identify the defect information included in each sample product image, each sample product image including at least one defect information; detect whether the defect represented by each defect information exists on the corresponding product; if so, determine the defect information as first sample defect information, and annotate the first sample defect information with a first label representing the existence of the defect; if not, determine the defect information as second sample defect information, and annotate the second sample defect information with a second label representing the absence of the defect.

[0138] In one embodiment, distribution of each piece of first sample defect information and each piece of second sample defect information included in the sampled defect set satisfies a Gaussian distribution.

[0139] In one embodiment, the model to be trained includes at least two classifiers;

[0140] a model training module, specifically configured to resample the plurality of first sample defect information and the plurality of second sample defect information at least twice based on the model adjustment parameters, wherein the plurality of sample defect information obtained after each resampling constitutes a sample defect set, and each sample defect set corresponds to a classifier;

[0141] The model training module is specifically used to input the sample defect information of each sample defect set into the to-be-trained model to obtain the classification result corresponding to the sample defect information.

[0142] In one embodiment, the model determination module 503 is specifically configured to update the standard defect detection model based on the target model adjustment parameters to obtain the target defect detection model.

[0143] In one embodiment, the model determination module 503 is specifically used to resample multiple first sample defect information and multiple second sample defect information at least twice based on the target model adjustment parameters. The multiple sampled sample defect information obtained after each resampling constitutes a sample defect set, and each sample defect set corresponds to a classifier; for each sample defect set, the sample defect information of the sample defect set is input into the standard defect detection model to obtain the detection result corresponding to the sample defect information; based on the detection result and the label of the sample defect information, the loss function value of the standard defect detection model is determined; if the loss function value is less than or equal to the second preset loss threshold or the current iteration reaches the second preset number of iterations, the standard defect detection model is determined as the target defect detection model; otherwise, new sample defect information in the sample defect set is obtained, and the step of inputting the sample defect information of the sample defect set into the standard defect detection model is returned.

[0144] In one embodiment, the model determination module 503 is specifically configured to determine the standard defect detection model corresponding to the target model adjustment parameter as the target defect detection model based on the correspondence between each parameter and each standard defect detection model.

[0145] In one embodiment, the detection requirement information and the target model adjustment parameter satisfy the following requirement: the lower the detection rate requirement for defects within the product specifications represented by the detection requirement information, the larger the target model adjustment parameter.

[0146] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0147] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0148] like Figure 6As shown, electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of device 600 can also be stored in RAM 603. Computing unit 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0149] Multiple components in the electronic device 600 are connected to the I / O interface 605, including an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0150] Computing unit 601 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 601 performs the various methods and processes described above, such as the defect detection method. For example, in some embodiments, the defect detection method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps of the defect detection method described above can be performed. Alternatively, in other embodiments, computing unit 601 can be configured to perform the defect detection method in any other suitable manner (e.g., via firmware).

[0151] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0156] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0157] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0158] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0159] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A defect detection method, characterized in that: The method comprises: Obtain images of products to be inspected and inspection requirement information; Determining target model adjustment parameters according to the detection requirement information; Determining a target defect detection model based on the target model adjustment parameters and a standard defect detection model; the standard defect detection model is obtained by pre-training a to-be-trained model based on first sample defect information, second sample defect information, a first label of the first sample defect information, and a second label of the second sample defect information, wherein the first sample defect information is information indicating the presence of a defect, and the second sample defect information is information indicating the absence of a defect; The image of the product to be inspected is input into the target defect detection model to obtain a defect detection result.

2. The method according to claim 1, characterized in that The training process of the standard defect detection model includes: Obtain multiple sample product images; determining, based on the plurality of sample product images, a plurality of first sample defect information and a plurality of second sample defect information, a first label for the first sample defect information and a second label for the second sample defect information; resampling the plurality of first sample defect information and the plurality of second sample defect information based on the model adjustment parameter to obtain a sample defect set consisting of the plurality of sampled sample defect information; Inputting the sample defect information of the sample defect set into the to-be-trained model to obtain the classification results corresponding to the sample defect information; Determining a loss function value of the to-be-trained model based on the classification result and the label of the sample defect information; If the loss function value is less than or equal to the first preset loss threshold or the current iteration reaches the first preset number of iterations, the model to be trained is determined as the standard defect detection model; otherwise, new sample defect information in the sample defect set is obtained, and the step of inputting the sample defect information of the sample defect set into the model to be trained is returned to execute, and the standard defect detection model corresponds to the model adjustment parameters.

3. The method according to claim 2, characterized in that The determining, based on the plurality of sample product images, a plurality of first sample defect information and a plurality of second sample defect information, a first label of the first sample defect information and a second label of the second sample defect information includes: For each sample product image, identifying defect information included in the sample product image, each sample product image including at least one piece of defect information; Detect whether the defect represented by each defect information exists on the corresponding product; If so, determining the defect information as first sample defect information, and annotating the first sample defect information with a first tag indicating the presence of the defect; If not, the defect information is determined as second sample defect information, and a second label indicating that the defect does not exist is marked on the second sample defect information.

4. The method according to claim 2, characterized in that The distribution of each piece of first sample defect information and each piece of second sample defect information included in the sampled defect set satisfies a Gaussian distribution.

5. The method according to claim 2, characterized in that The model to be trained includes at least two classifiers; The resampling of the plurality of first sample defect information and the plurality of second sample defect information based on the model adjustment parameter to obtain a sample defect set consisting of the plurality of sampled sample defect information includes: resampling the plurality of first sample defect information and the plurality of second sample defect information at least twice based on the model adjustment parameters, wherein the plurality of sample defect information obtained after each resampling constitutes a sample defect set, and each sample defect set corresponds to a classifier; The step of inputting the sample defect information of the sample defect set into the to-be-trained model to obtain the classification result corresponding to the sample defect information includes: For each sampled defect set, the sample defect information of the sampled defect set is input into the model to be trained to obtain the classification result corresponding to the sample defect information.

6. The method according to claim 1, wherein The determining of the target defect detection model based on the model adjustment parameters and the standard defect detection model includes: The standard defect detection model is updated based on the target model adjustment parameters to obtain a target defect detection model.

7. The method according to claim 6, characterized in that The updating of the standard defect detection model based on the target model adjustment parameters to obtain the target defect detection model includes: resampling the plurality of first sample defect information and the plurality of second sample defect information at least twice based on the target model adjustment parameters, wherein the plurality of sample defect information obtained after each resampling constitutes a sample defect set, and each sample defect set corresponds to a classifier; For each sampled defect set, input the sample defect information of the sampled defect set into the standard defect detection model to obtain the detection results corresponding to the sample defect information; Determining a loss function value of the standard defect detection model based on the detection result and the label of the sample defect information; If the loss function value is less than or equal to the second preset loss threshold or the current iteration reaches the second preset number of iterations, the standard defect detection model is determined as the target defect detection model; otherwise, new sample defect information in the sample defect set is obtained, and the step of inputting the sample defect information of the sample defect set into the standard defect detection model is returned to execute.

8. A defect detection device, characterized in that: The device comprises: Information acquisition module, used to obtain images of products to be inspected and inspection requirement information; A parameter determination module, configured to determine target model adjustment parameters according to the detection requirement information; a model determination module, configured to determine a target defect detection model based on the target model adjustment parameters and a standard defect detection model; the standard defect detection model is obtained by pre-training a to-be-trained model based on first sample defect information, second sample defect information, a first label of the first sample defect information, and a second label of the second sample defect information, wherein the first sample defect information is information indicating the presence of a defect, and the second sample defect information is information indicating the absence of a defect; The defect detection module is used to input the image of the product to be detected into the target defect detection model to obtain a defect detection result.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images