Model training method and device, image detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a model training method and device, an image detection method and device, electronic equipment and a storage medium, and the model training method comprises the steps: constructing a first model; the first model is obtained by adding a first network, a second network, a third network and a fourth network to the second model; the second model represents a defect detection model which completes training and is used for carrying out defect detection on the industrial component in the input image based on the convolutional features of the input image; training the first model based on the plurality of first training samples until a set convergence condition is reached; wherein the model parameters of the second model remain unchanged in the training process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a model training method and image detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] In related technologies, a single model is used to detect defects of industrial parts in an input image to identify image content corresponding to the defects of the industrial parts in the input image. However, the observation angles of the industrial parts in the input image are diverse, and the pixel proportion of the image content corresponding to the defects of the industrial parts in the image is small, which results in low accuracy of defect detection. SUMMARY

[0003] To solve the problems in related technologies, the present application provides a model training method and image detection method and device, electronic equipment and storage medium.

[0004] The technical solution of the present application embodiment is implemented as follows:

[0005] The present application embodiment provides a model training method, which comprises:

[0006] constructing a first model; the first model is obtained by adding a first network, a second network, a third network and a fourth network to a second model; the second model represents a defect detection model that has been trained and is used to detect defects of industrial parts in an input image based on convolution features of the input image; the first network is used to determine the defect degree of the industrial parts in the input image based on the convolution features of the input image; the second network is used to perform feature dimension transformation on the convolution features of the input image; the third network is used to extract convolution features of a reflection image of the input image; and the fourth network is used to determine the defect detection result of the industrial parts in the input image based on the outputs of the first network, the second network and the third network;

[0007] training the first model based on a plurality of first training samples until a set convergence condition is reached; wherein the model parameters of the second model remain unchanged during the training process.

[0008] In the above solution, the method further comprises:

[0009] calling the second model to process a plurality of first input images to obtain a defect detection result of each first input image;

[0010] constructing the plurality of first training samples based on a plurality of second input images, wherein the second input images represent first input images with defects.

[0011] In the scheme, the method further comprises:

[0012] The sliding window processing is performed in each second input image, and the proportion of first pixels in each sliding window coverage area in the second input image is counted, the first pixels representing pixels labeled as defects in the second input image;

[0013] A first output matrix is constructed based on the proportion of the first pixels in each sliding window coverage area; wherein,

[0014] The first training sample represents the second input image, and the label corresponding to the first training sample represents the first output matrix.

[0015] In the scheme, the method further comprises:

[0016] When the first model is trained based on a plurality of first training samples, a first loss function is called to process the second output matrix corresponding to each positive sample in the plurality of first training samples and the first output matrix corresponding to the positive sample, to obtain a first loss value, and the first loss function is called to process the second output matrix corresponding to each negative sample in the plurality of first training samples and the first output matrix corresponding to the negative sample, to obtain a second loss value; the second output matrix represents an output matrix obtained by calling the first model to process the first training sample;

[0017] The second loss value and the first product are summed to obtain the third loss value; the first product represents the product of the first weight and the first loss value; the first weight represents the ratio between the number of negative samples and the number of positive samples in the plurality of first training samples, and the third loss value is used to determine whether the set convergence condition is reached.

[0018] In the scheme, the method further comprises:

[0019] The defect detection model is trained based on a plurality of second training samples to obtain the second model; wherein,

[0020] The second training sample represents an image of an industrial part without defects.

[0021] In the scheme, the second model comprises a teacher network encoder and a student network decoder running based on a reverse distillation mechanism, and correspondingly,

[0022] The first network is used to output a first feature matrix based on the feature distance between the first convolutional features output by the teacher network encoder and the second convolutional features output by the student network decoder, the first feature matrix representing the defect degree of the industrial part in the input image;

[0023] The second network is configured to merge the first convolutional features of each layer output by the teacher network encoder, and perform dimension reduction processing on the merged third convolutional features.

[0024] In the above scheme, the third network is further configured to obtain the reflection image based on Gaussian convolution calculation on the input image.

[0025] The embodiments of the present application further provide an image detection method, which comprises:

[0026] calling a third model to process a third input image to obtain a third output matrix, the third output matrix representing a defect detection result of an industrial part contained in image content of the third input image, and the third input image representing an image containing the image content of the industrial part;

[0027] The third model represents the first model trained based on any of the above model training methods.

[0028] In the above scheme, the method further comprises:

[0029] performing binary conversion on each element in the third output matrix to obtain a fourth output matrix;

[0030] performing adjacent interpolation calculation on the fourth output matrix to obtain a first output image;

[0031] performing connected domain calculation on the first output image to obtain a positioning frame, and the positioning frame is configured to frame the image content corresponding to the defect of the industrial part in the third input image.

[0032] The embodiments of the present application further provide a model training device, which comprises:

[0033] a construction unit configured to construct a first model, the first model being obtained by adding a first network, a second network, a third network and a fourth network to a second model, the second model representing a defect detection model that has been trained and is configured to detect defects of an industrial part in an input image based on convolutional features of the input image, the first network being configured to determine a defect degree of the industrial part in the input image based on the convolutional features of the input image, the second network being configured to perform feature dimension transformation on the convolutional features of the input image, the third network being configured to extract convolutional features of a reflection image of the input image, and the fourth network being configured to determine a defect detection result of the industrial part in the input image based on outputs of the first network, the second network and the third network;

[0034] a training unit configured to train the first model based on a plurality of first training samples until a set convergence condition is reached, wherein model parameters of the second model remain unchanged during the training process.

[0035] The embodiment of the present application also provides an image detection device, comprising:

[0036] The first calling unit is configured to call the third model to process a third input image to obtain a third output matrix, wherein the third output matrix represents a defect detection result of an industrial component contained in image content of the third input image, and the third input image represents an image containing the image content of the industrial component.

[0037] The third model represents a first model trained based on any of the foregoing model training methods.

[0038] The embodiment of the present application also provides an electronic device, comprising a first processor and a first communication interface.

[0039] The first processor is configured to construct a first model, wherein the first model is obtained by adding a first network, a second network, a third network and a fourth network to a second model; the second model represents a defect detection model that has been trained and is used to detect defects of an industrial component in an input image based on a convolution feature of the input image; the first network is used to determine a defect degree of the industrial component in the input image based on the convolution feature of the input image; the second network is used to perform feature dimension transformation on the convolution feature of the input image; the third network is used to extract a convolution feature of a reflection image of the input image; and the fourth network is used to determine a defect detection result of the industrial component in the input image based on outputs of the first network, the second network and the third network; and

[0040] The first model is trained based on a plurality of first training samples until a set convergence condition is reached; wherein model parameters of the second model remain unchanged during the training process.

[0041] The embodiment of the present application also provides an electronic device, comprising a second processor and a second communication interface.

[0042] The second processor is configured to call a third model to process a third input image to obtain a third output matrix, wherein the third output matrix represents a defect detection result of an industrial component contained in image content of the third input image, and the third input image represents an image containing the image content of the industrial component.

[0043] The third model represents a first model trained based on any of the foregoing model training methods.

[0044] The embodiment of the present application also provides an electronic device, comprising a first processor and a first memory for storing a computer program capable of running on the processor,

[0045] The first processor is configured to execute the steps of any of the preceding model training methods when running the computer program.

[0046] The embodiments of the present application also provide an electronic device, comprising a second processor and a second memory for storing a computer program capable of running on the processor,

[0047] The second processor is configured to execute the steps of any of the preceding image detection methods when running the computer program.

[0048] The embodiments of the present application also provide a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the preceding methods.

[0049] The embodiments of the present application also provide a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the preceding methods.

[0050] In the embodiments of the present application, a first model is constructed, wherein the first model is obtained by adding a first network, a second network, a third network and a fourth network to a second model, the second model represents a defect detection model that has been trained and is used for detecting defects of an industrial part in an input image based on a convolution feature of the input image, the first network is used for determining a defect degree of the industrial part in the input image based on the convolution feature of the input image, the second network is used for performing feature dimension transformation on the convolution feature of the input image, the third network is used for extracting a convolution feature of a reflection image of the input image, and the fourth network is used for determining a defect detection result of the industrial part in the input image based on outputs of the first network, the second network and the third network; then, the first model is trained based on a plurality of first training samples until a set convergence condition is reached, wherein model parameters of the second model remain unchanged during the training process. In the above scheme, the first model is obtained by adding a plurality of networks to the second model, that is, a plurality of network models are fused in the first model, and the defect detection result output by the first model is determined based on outputs of the first network, the second network and the third network in the fourth network of the first model. On this basis, after the training of the first model is completed, the trained first model can be called to detect defects of the industrial part in the input image. Compared with the related art, the defect detection result can be obtained based on outputs of a plurality of network models when the defect is detected, so that the feature information used to determine the defect detection result is more abundant, thereby improving the accuracy of defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 An implementation flowchart of a model training method provided by the embodiments of the present application is shown in the figure;

[0052] Figure 2A schematic diagram of a reflected image provided for an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a first training sample provided for an embodiment of the present application;

[0054] Figure 4 A schematic diagram of a dimension reduction module provided for an embodiment of the present application;

[0055] Figure 5 A schematic diagram of a feature extraction network provided for an embodiment of the present application;

[0056] Figure 6 A schematic diagram of a fourth network provided for an embodiment of the present application;

[0057] Figure 7 A schematic diagram of a multi-source feature fusion defect detection model provided for an embodiment of the present application;

[0058] Figure 8 An implementation flowchart of an image detection method provided for an embodiment of the present application;

[0059] Figure 9 A positioning frame schematic diagram provided for an embodiment of the present application;

[0060] Figure 10 A structural schematic diagram of a model training device provided for an embodiment of the present application;

[0061] Figure 11 A structural schematic diagram of an image detection device provided for an embodiment of the present application;

[0062] Figure 12 A hardware composition structural schematic diagram of an electronic device provided for an embodiment of the present application;

[0063] Figure 13 A hardware composition structural schematic diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0064] In an industrial scene, an industrial part is imaged, for example, a camera is used to take an image of the industrial part to obtain an image containing the industrial part in the image content, and then the detection of the industrial part in the image is realized through image detection.

[0065] In the related art, a single model is used to detect defects of an industrial part in an input image to identify image content corresponding to the defects of the industrial part in the input image. For example, the single model can be a patchcore model aiming at full recall rate. However, the observation angle of the industrial part in the input image is diverse, the image content is complex, and the pixel proportion of the image content corresponding to the defects of the industrial part in the image is small due to the yield requirement of the industrial part. Therefore, the single model is prone to false positives when detecting defects of the industrial part in the input image, resulting in low accuracy of defect detection.

[0066] Therefore, in the embodiments of the present application, a first model is constructed, wherein the first model is obtained by adding a first network, a second network, a third network and a fourth network to a second model. The second model represents a defect detection model that has been trained and is used to detect defects of an industrial part in an input image based on the convolution features of the input image. The first network is used to determine the defect degree of the industrial part in the input image based on the convolution features of the input image. The second network is used to perform feature dimension transformation on the convolution features of the input image. The third network is used to extract the convolution features of a reflection image of the input image. The fourth network is used to determine the defect detection result of the industrial part in the input image based on the outputs of the first network, the second network and the third network. Then, the first model is trained based on a plurality of first training samples until a set convergence condition is reached, wherein the model parameters of the second model remain unchanged during the training process. In the above scheme, the first model is obtained by adding a plurality of networks to the second model, that is, a plurality of network models are fused in the first model, and the defect detection result output by the first model is determined by the fourth network in the first model based on the outputs of the first network, the second network and the third network. On this basis, after the training of the first model is completed, the trained first model can be called to detect defects of the industrial part in the input image. Compared with the related art, the defect detection result can be obtained based on the outputs of a plurality of network models when defect detection is performed, so that the feature information used to determine the defect detection result is more abundant, thereby improving the accuracy of defect detection.

[0067] The present application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0068] The embodiments of the present application provide a model training method, referring to Figure 1 The method comprises the following steps:

[0069] Step 101: Construct a first model.

[0070] The first model is obtained by adding a first network, a second network, a third network and a fourth network on the second model; the second model represents a defect detection model that has been trained and is used for defect detection of an industrial part in an input image based on a convolution feature of the input image; the first network is used for determining a defect degree of the industrial part in the input image based on the convolution feature of the input image; the second network is used for performing feature dimension transformation on the convolution feature of the input image; the third network is used for extracting a convolution feature of a reflection image of the input image; and the fourth network is used for determining a defect detection result of the industrial part in the input image based on outputs of the first network, the second network and the third network.

[0071] Here, the first model is obtained by adding the first network, the second network, the third network and the fourth network on the second model, which can be understood as that the first model fuses the second model, the first network, the second network, the third network and the fourth network, and the second model and the networks can be understood as network models.

[0072] Here, the second model has been trained, and in actual application, the second model can be a defect detection model that has been put into use in an online environment. Exemplarily, the second model can be a patchcore model or a teacher-student model.

[0073] In actual application, the first model after training can be put into use after the second model, and the defect detection capability of the first model after training is superior to that of the second model.

[0074] Here, the first network is used for determining the defect degree of the industrial part in the input image based on the convolution feature of the input image, and the second network is used for performing feature dimension transformation on the convolution feature of the input image. In actual application, the convolution features processed by the first network and the second network can be determined based on an output obtained by calling the second model to process the input image.

[0075] Here, the third network is used for extracting the convolution feature of the reflection image of the input image. In actual application, the reflection image of the input image can be composed of image contents that can remain unchanged under different light conditions, that is, the reflection image can be regarded as an image obtained by removing highlights from the input image.

[0076] Exemplarily, Figure 2 A reflection image example is provided, in which the right image can be regarded as the reflection image of the left image.

[0077] In actual application, the reflection image weakens the influence of environmental light compared with the input image, and therefore, the convolution feature of the reflection image can better reflect the attention of the model to the object in the image, which helps to improve the accuracy of defect detection of the industrial part in the image.

[0078] The fourth network is configured to determine the defect detection result of the industrial part in the input image based on the outputs of the first network, the second network, and the third network. In actual applications, the outputs of the first network, the second network, and the third network can be understood as features of the input image. The fourth network can fuse the features output by the first network, the second network, and the third network to obtain fused features, and then determine the defect detection result of the industrial part in the input image based on the fused features.

[0079] In actual applications, the fused features contain richer feature information than the features output by a single model, which can provide more sufficient information reference for the defect detection of the industrial part in the image, thereby improving the accuracy of the defect detection result.

[0080] Step 102: training the first model based on a plurality of first training samples until a set convergence condition is reached.

[0081] In the training process, the model parameters of the second model remain unchanged.

[0082] In actual applications, the first model can be trained multiple times based on a plurality of first training samples until a set convergence condition is reached. After each training iteration is completed, the model parameters of each model and network in the first model can be updated. The model parameters can be understood as configuration variables inside the model, which can be the weights and biases of each node of the model.

[0083] In actual applications, the set convergence condition can be that the calculated loss value is less than a set threshold. The loss value can be calculated in each training iteration. The loss value can be used to describe the degree of inconsistency between the predicted value obtained based on the training sample and the target value expected to be obtained based on the training sample.

[0084] In actual applications, in the process of one of the multiple training iterations of the first model, the second model can be called to process the first training sample to obtain the convolutional features corresponding to the first training sample. Then, the first network is called to process the convolutional features output by the second model, the second network is called to process the convolutional features output by the second model, and the third network is called to extract the convolutional features of the reflection image corresponding to the first training sample. Then, the fourth network is called to process the outputs of the first network, the second network, and the third network to obtain the defect detection result of the industrial part in the image corresponding to the first training sample, i.e., the predicted value. Then, the loss value is calculated based on the predicted value, and the model parameters of each model and network in the first model are updated based on the loss value, and it is determined whether the set convergence condition is reached.

[0085] Here, the model parameters of the second model remain unchanged during the training process, that is, the model parameters of the second model are not updated during the multiple training iterations of the first model.

[0086] In actual applications, the first model can obtain a third model after the training is completed, and the third model can be understood as the first model after the training is completed. After obtaining the third model, the third model can be called to process the image of the industrial part to be defectively detected to obtain the corresponding defect detection result.

[0087] In the embodiments of the present application, the first model is obtained by adding the first network, the second network and the third network on the second model, that is, the first model integrates multiple network models, and the defect detection result output by the first model is determined by the fourth network in the first model based on the outputs of the first network, the second network and the third network. On this basis, after the training of the first model is completed, the trained first model can be called to perform defect detection on the industrial part in the input image. Compared with the related art, the defect detection result can be obtained based on the outputs of multiple network models when performing defect detection, so that the feature information used to determine the defect detection result is more abundant, thereby improving the accuracy of defect detection.

[0088] The training process of the first model is further described below.

[0089] In actual applications, due to the yield requirement of the industrial part, the image of the industrial part with defects is accumulated slowly, and it is not easy to quickly build the training sample of the image of the industrial part with defects. Based on this,

[0090] In an embodiment, the model training method provided by the embodiments of the present application further includes:

[0091] training the defect detection model based on the plurality of second training samples to obtain a second model; wherein

[0092] The second training sample represents an image of an industrial part without defects.

[0093] Here, the defect detection model is trained based on the image of the industrial part without defects to obtain the second model, which avoids the problem of insufficient training samples caused by the slow accumulation of the image of the industrial part with defects. In this way, the second model can be quickly put into operation, improving the processing efficiency of defect detection, so as to meet the urgent demand for defect detection function in the industrial scene.

[0094] In actual applications, after the second model is put into operation, the second model can be called to perform defect detection on the image of the industrial part to be defectively detected online to accumulate the image of the industrial part with defects, and then the accumulated image is used to train the first model. Based on this,

[0095] In an embodiment, the model training method provided by the embodiments of the present application further includes:

[0096] calling the second model to process the plurality of first input images to obtain a defect detection result of each first input image;

[0097] constructing a plurality of first training samples based on a plurality of second input images, the second input image representing a corresponding defect detection result as a first input image with defects.

[0098] In actual application, the second input image can be understood as an image of an industrial part with defects accumulated after the second model goes online, or as a historical processing image of the second model.

[0099] In the embodiments of the present application, a plurality of first training samples are constructed based on a plurality of second input images, so that the first model can be trained based on images of industrial parts with defects, improving the training effect of the first model, thereby enhancing the processing capability of the first model after training and improving the defect detection accuracy.

[0100] In actual application, in the process of constructing the first training sample, the second input image can be preprocessed, and then the first training sample is constructed based on the preprocessed second input image; for example, the preprocessing can include adding random angle rotation and brightness change, adjusting the size, and normalizing the image. In this way, the diversity of the training sample can be increased, thereby improving the robustness of the first model after training.

[0101] In actual application, the defect detection result corresponding to the second input image output by the second model can also be used to determine the label corresponding to the first training sample.

[0102] In an embodiment, the model training method provided by the embodiments of the present application further includes:

[0103] performing a sliding window processing in each second input image and counting the proportion of first pixels in each sliding window coverage area in the second input image, the first pixel representing a pixel labeled as a defect in the second input image;

[0104] constructing a first output matrix based on the proportion of the first pixels in each sliding window coverage area; wherein,

[0105] the first training sample represents the second input image, and the label corresponding to the first training sample represents the first output matrix.

[0106] In actual application, the defect detection result corresponding to the second input image output by the second model can be regarded as a pixel-level label of the defect in the second input image, and the first pixel can be determined based on the defect detection result output by the second model. The defect in the image can be understood as the image content corresponding to the defect of the industrial part.

[0107] In actual application, during the sliding window processing of the second input image, the proportion of the first pixel in each sliding window coverage area can be counted, which can be regarded as the probability true value of containing defects in the sliding window coverage area; and then the first output matrix is constructed based on the proportion corresponding to each sliding window coverage area. Each element in the first output matrix can correspond to a sliding window coverage area.

[0108] In actual application, after the first output matrix corresponding to the second input image is constructed, the first output matrix can be used as the label corresponding to the first training sample of the second input image. The label corresponding to the first training sample can be understood as the target value expected to be obtained by the first model based on the first training sample.

[0109] Here, the first training sample is represented by the second input image, and in actual application, the first training sample can also be constructed based on the images of multiple defect-free industrial parts. The construction process of the first training sample corresponding to the image of the defect-free industrial part can be understood in a corresponding manner with reference to the construction process of the first training sample corresponding to the second input image. The first training sample constructed based on the image of the defect-free industrial part can be represented as the image of the defect-free industrial part. In this way, the diversity of the first training sample can be improved, and the robustness of the first model after training can be further improved.

[0110] In actual application, calling the first model to process the first training sample can be understood as calling the first model to process the input image or the preprocessed input image corresponding to the first training sample.

[0111] In actual application, the first training sample corresponding to an input image can also include a reflection image corresponding to the input image. In this way, in the process of extracting the convolutional features of the reflection image of the input image based on the first training sample, the third network in the first model can directly extract the reflection image included in the first training sample. It should be noted that the first training sample corresponding to the input image can also not include the reflection image. In this case, the third network can process the input image or the preprocessed input image corresponding to the first training sample to obtain the reflection image, and then extract the convolutional features of the reflection image.

[0112] For ease of understanding, Figure 3A first training sample example is provided, where input1 (input 1) corresponds to an input image corresponding to the first training sample, and the image size is 1024x1024; input2 (input 2) corresponds to the reflection image of the input image corresponding to the first training sample, and the image size is 1024x1024; target (target value) corresponds to the label corresponding to the first training sample, and the image size is 256x256.

[0113] In an embodiment, the model training method provided by the embodiment of the application further includes:

[0114] When training the first model based on the plurality of first training samples, the first loss function is called to process the second output matrix corresponding to each positive sample in the plurality of first training samples and the first output matrix corresponding to the positive sample, to obtain a first loss value, and the first loss function is called to process the second output matrix corresponding to each negative sample in the plurality of first training samples and the first output matrix corresponding to the negative sample, to obtain a second loss value; the second output matrix represents an output matrix obtained by calling the first model to process the first training sample;

[0115] The second loss value and the first product are summed to obtain a third loss value; the first product represents the product of the first weight and the first loss value; the first weight represents the ratio between the number of negative samples and the number of positive samples in the plurality of first training samples, and the third loss value is used to determine whether a set convergence condition is reached.

[0116] In actual application, the positive sample can be understood as the first training sample corresponding to the image of the industrial part with defects, and the negative sample can be understood as the first training sample corresponding to the image of the industrial part without defects. The first output matrix can be understood as the target value expected to be obtained by the first model based on the training sample, and the second output matrix can be understood as the prediction value obtained by the first model based on the training sample.

[0117] In actual application, the first loss function can be a cross-entropy loss function.

[0118] For example, the third loss value can be represented as:

[0119] loss=weighted×pos_bce_loss+neg_bac_loss,

[0120] Wherein, loss represents the third loss value, pos_bce_loss represents the first loss value, neg_bac_loss represents the second loss value, and weighted represents the first weight.

[0121] For example, the first weight can be represented as:

[0122] weighted=num_neg / num_pos,

[0123] wherein, num_neg represents the number of negative samples in the plurality of first training samples, and num_pos represents the number of positive samples in the plurality of first training samples.

[0124] In actual application, after obtaining the third loss value, gradient calculation can be performed based on the third loss value, and the model parameters in the first model can be updated. For example, the third loss value can be processed based on a Stochastic Gradient Descent (SDG) optimizer to perform gradient calculation and model parameter updating, and the learning rate can be adjusted based on a cosine annealing strategy during the training process.

[0125] In the embodiments of the present application, the first weight is obtained based on the number of negative samples and the number of positive samples in the plurality of first training samples, and then the third loss value is constructed based on the first weight. In this way, the training effect of the first model can be improved, thereby enhancing the processing capability of the first model after training and improving the accuracy of defect detection.

[0126] The following further describes each network model in the first model.

[0127] In an embodiment, the second model includes a teacher network encoder and a student network decoder running based on a reverse distillation mechanism, and correspondingly,

[0128] The first network is configured to output a first feature matrix based on the feature distance between the first convolutional features output by the teacher network encoder and the second convolutional features output by the student network decoder, and the first feature matrix represents the defect degree of the industrial part in the input image.

[0129] The second network is configured to merge the first convolutional features of each layer output by the teacher network encoder, and then perform dimension reduction processing on the third convolutional features obtained by the merging.

[0130] In actual application, the second model can be called to process the first training sample to obtain a plurality of convolutional features. The convolutional features output by the second model can be understood as the convolutional features of the input image corresponding to the first training sample.

[0131] In practical applications, the teacher network encoder in the second model can include multiple processing layers, each of which can output a first convolutional feature based on the input image. The student network decoder in the second model has the same number of processing layers as the teacher network encoder, and each of which can output a second convolutional feature based on the input image. The teacher network encoder and the student network decoder can be associated based on a One-Class Bottleneck Embedding (OCBE) module.

[0132] In practical applications, the first network can be invoked to process multiple first convolutional features and multiple second convolutional features output by the second model based on the first training samples, thereby obtaining the first feature matrix. Specifically,

[0133] During the process of calling the first network for processing, the first network can calculate the feature distance between the first convolutional feature output by each layer of the teacher network encoder and the second convolutional feature output by the corresponding layer of the student network decoder, and obtain one or more feature distances, wherein each feature distance corresponds to a processing layer of the teacher network encoder and also corresponds to a processing layer of the student network decoder.

[0134] For example, the feature distance can be represented as:

[0135] diff_score = 1 - cos(f) s ,f t ),

[0136] Where, diff_score represents the feature distance, f t Characterizing the first convolutional feature, f s Characterizing the second convolution feature, cos(f s ,f t The cosine similarity between the first convolutional feature and the second convolutional feature is represented.

[0137] After obtaining one or more feature distances, the first network can perform interpolation on each feature distance to process these feature distances to the same data size, and then merge the one or more interpolated feature distances in the channel dimension to obtain the first feature matrix.

[0138] Here, the first feature matrix is ​​obtained based on the feature distance between the first convolutional feature and the second convolutional feature. In practical applications, the first feature matrix can also be expressed as the anomaly distance feature.

[0139] In practical applications, a second network can be invoked to transform the feature dimensions of the first convolutional features of each layer output by the teacher network encoder, resulting in the transformed first convolutional features. Specifically,

[0140] In the process of calling the second network for processing, the second network can perform interpolation processing on the first convolutional features of each layer output by the teacher network encoder to process the first convolutional features into the same feature size, then merge one or more interpolated first convolutional features in the channel dimension to obtain third convolutional features, and then reduce the dimension of the third convolutional features in the channel dimension to obtain the first convolutional features after the dimension of the features is transformed.

[0141] In practical applications, the second network can include a dimension reduction network, which can also be referred to as a dimension reduction module. For example, referring to Figure 4 , the dimension reduction module can include three convolutional computing structures and a residual structure to realize feature dimension reduction operations and information transmission during the operations.

[0142] In practical applications, the third network can be called to process the first training sample to obtain the convolutional features of the reflection image of the input image corresponding to the first training sample.

[0143] In an embodiment, the third network is further configured to obtain the reflection image based on Gaussian convolution calculation on the input image.

[0144] In practical applications, Gaussian convolution calculation can be performed on the input image based on the Retinex theory to obtain the reflection image.

[0145] For example, the calculation process of the third network to obtain the reflection image of the input image based on the input image can be represented as:

[0146] Log(R(x, y)) = Log(I(x, y)) - Log(I(x, y) x G(x, y)),

[0147] where I(x, y) represents the input image, I(x, y) x G(x, y) represents Gaussian convolution calculation on the input image, and R(x, y) represents the reflection image.

[0148] In practical applications, the data represented by Log(R(x, y)) calculated can be quantized to 0 to 255 to obtain the reflection image.

[0149] In practical applications, the third network can include a feature extraction network, which can be used to extract the convolutional features of the reflection image. For example, referring to Figure 5 , the feature extraction network in the third network can be stacked with convolutional computing structures and residual structures.

[0150] In actual application, the fourth network can be called to process the outputs of the first network, the second network and the third network to obtain the defect detection result of the industrial part in the input image corresponding to the first training sample.

[0151] In actual application, the fourth network can include a feature fusion network and a classification network. In the process of calling the fourth network for processing, the feature fusion network can fuse the outputs of the first network, the second network and the third network to obtain fused features, and the classification network can process the fused features to obtain the corresponding defect detection result. The classification network can also be referred to as a classification module.

[0152] For example, the classification module can include an instance norm, a convolution calculation structure (conv) and a sigmoid activation structure. The sigmoid activation structure can be used to determine the probability that each pixel position of the input image contains image content corresponding to the defect of the industrial part.

[0153] For example, Figure 6 An example of the structure of the fourth network is provided, in which the abnormal distance feature corresponds to the output of the first network, the original image feature corresponds to the output of the second network, and the reflection image feature corresponds to the output of the third network.

[0154] In the embodiments of the present application, the first model is obtained by adding multiple networks to the second model, that is, the first model integrates multiple network models, and the defect detection result output by the first model is determined by the fourth network in the model structure based on the outputs of the first network, the second network and the third network. On this basis, after the training of the first model is completed, the trained first model can be called to detect defects of the industrial part in the input image. Compared with the related art, the defect detection result can be obtained based on the outputs of multiple network models when defect detection is performed, so that the feature information used to determine the defect detection result is more abundant, thereby improving the accuracy of defect detection.

[0155] For ease of understanding, Figure 7 An example of the structure of the multi-source feature fusion defect detection model is provided, in which the multi-source feature fusion defect detection model corresponds to the first model in the embodiments of the present application before training, the fixed-weight anomaly detection model corresponds to the second model in the embodiments of the present application, the image feature processing module corresponds to the first network in the embodiments of the present application, the abnormal score feature extraction module corresponds to the second network in the embodiments of the present application, the reflection feature extraction module corresponds to the third network in the embodiments of the present application, and the feature fusion classification module corresponds to the fourth network in the embodiments of the present application.

[0156] In actual application, the fourth network can be called to process the outputs of the first network, the second network and the third network to obtain the defect detection result of the industrial part in the input image corresponding to the first training sample. Figure 7The training performed by the multi-source feature fusion defect detection model in the method can mainly include the following steps:

[0157] Step 1: calling the fixed-weight anomaly detection model to process the input image to obtain a plurality of first convolutional features output by a teacher network encoder and a plurality of second convolutional features output by a student network decoder.

[0158] In actual application, the input image in the training stage can be understood as an input image corresponding to a first training sample, and the first training sample can be constructed based on historical processing images of the fixed-weight anomaly detection model that has been put into operation.

[0159] Step 2: calling an image feature processing module to process the plurality of first convolutional features to obtain first convolutional features after feature dimension transformation.

[0160] Step 3: calling an anomaly score feature extraction module to process the plurality of first convolutional features and the plurality of second convolutional features to obtain a first feature matrix.

[0161] Step 4: calling a reflection map feature extraction module to process a reflection image of the input image to obtain a convolutional feature of the reflection image of the input image.

[0162] In actual application, the reflection map feature extraction module can also be called to process the input image to extract the reflection image of the input image, and further extract the convolutional feature of the reflection image.

[0163] Step 5: calling a feature fusion classification module to process the first convolutional features after feature dimension transformation, the first feature matrix, and the convolutional feature of the reflection image of the input image to obtain a defect detection result of an industrial component in the input image.

[0164] In actual application, the above steps 3, 4, and 5 can be executed all at once or partially, or can be executed step by step, and the execution order is not limited here. For example, steps 4 and 5 can be executed first, and then step 3.

[0165] Step 6: calculating an overall loss value based on the defect detection result obtained in step 5.

[0166] In actual application, the overall loss value is equivalent to the third loss value in the embodiments of the present application.

[0167] Step 7: updating the model parameters of the multi-source feature fusion defect detection model based on the overall loss value.

[0168] In actual application, in the process of updating the model parameters of the multi-source feature fusion defect detection model, the model parameters of the fixed-weight anomaly detection model remain unchanged.

[0169] Step 8: Repeat steps 1 to 7 until a set convergence condition is reached.

[0170] In actual applications, the multi-source feature fusion defect detection model is trained when the set convergence condition is reached.

[0171] Based on the model training method in the above embodiments, the embodiments of the present application also provide an image detection method, which is described below with reference to Figure 8 The method comprises the following steps.

[0172] Step 801: calling a third model to process a third input image to obtain a third output matrix.

[0173] The third output matrix represents a defect detection result of an industrial part contained in the image content of the third input image, and the third input image represents an image containing the image content of the industrial part.

[0174] The third model is trained based on the model training method in any of the above embodiments.

[0175] In actual applications, the third input image can be understood as an image of an industrial part to be defect detected.

[0176] In actual applications, the model structure in the third model can include the second model, the fifth network, the sixth network, the seventh network, and the eighth network. The fifth network can be regarded as the first network after training, the sixth network can be regarded as the second network after training, the seventh network can be regarded as the third network after training, and the eighth network can be regarded as the fourth network after training.

[0177] In actual applications, in the process of calling the third model to process the third input image, the second model can be called to process the third input image to obtain the convolutional features of the third input image, then the fifth network is called to process the convolutional features of the third input image, the sixth network is called to process the convolutional features of the third input image, and the seventh network is called to extract the convolutional features of the corresponding reflection image of the third input image, and then the eighth network is called to process the outputs of the sixth network, the seventh network, and the eighth network to obtain the third output matrix.

[0178] It should be noted that the third model is an end-to-end model, and the third model directly outputs the third output matrix based on the third input image; the intermediate output content involved in the third model, such as the output of the first network, is only used to explain the combination mode of the internal structure of the model, and does not mean the actual output of the third model.

[0179] In an embodiment, the image detection method provided by the embodiments of the present application further comprises:

[0180] binarizing each element in the third output matrix to obtain a fourth output matrix;

[0181] performing a neighborhood interpolation calculation on the fourth output matrix to obtain a first output image;

[0182] performing a connected component calculation on the first output image to obtain a positioning frame; the positioning frame is used to frame image content corresponding to the defect of the industrial part in the third input image.

[0183] In actual application, each element in the third output matrix can represent a probability that an image region of a set size in the third input image contains a defect, and the third output matrix can be understood as a probability matrix.

[0184] In actual application, each element in the third output matrix can be binarized based on a set threshold to filter out image regions with low probability, thereby obtaining the fourth output matrix, which can be represented as a binary matrix.

[0185] For example, assuming that the set threshold is 0.86, when each element in the third output matrix is binarized based on the set threshold, the value of an element with a value lower than 0.86 can be converted to 0, and the value of an element with a value higher than or equal to 0.86 can be converted to 1. In this way, the third output matrix obtained only includes elements with values of 1 or 0, wherein a value of 0 can represent that the corresponding image region does not contain a defect, and a value of 1 can represent that the corresponding image region contains a defect.

[0186] In actual application, the neighborhood interpolation calculation performed on the fourth output matrix can make the image size of the first output image the same as the image size of the third input image.

[0187] In actual application, after obtaining the first output image, a connected component calculation can be performed on the first output image to obtain a positioning frame, and the positioning frame can be marked in the third output image to frame image content corresponding to the defect of the industrial part in the third input image.

[0188] For ease of understanding, Figure 9 An example of the positioning frame is given, wherein the content framed by the rectangular frame is the image content corresponding to the defect of the industrial part.

[0189] In an embodiment of the present application, a third model is called to process the third input image, to obtain a defect detection result of an industrial part contained in the image content of the third input image, wherein the third model represents the trained first model, and the first model is obtained by adding multiple networks to the second model, that is, the first model integrates multiple network models, and the defect detection result output by the first model is determined by the fourth network in the model structure based on the outputs of the first network, the second network and the third network. On this basis, compared with the related art, the third model can obtain the defect detection result based on the outputs of multiple network models when performing defect detection, so that the feature information used to determine the defect detection result is more abundant, thereby improving the accuracy of defect detection.

[0190] Based on the above embodiments, the present application further provides a model training apparatus, which is described below with reference to Figure 10 The model training apparatus comprises:

[0191] A construction unit 1001 is configured to construct a first model, wherein the first model is obtained by adding a first network, a second network, a third network and a fourth network to a second model; the second model represents a trained defect detection model used to perform defect detection on an industrial part in an input image based on a convolution feature of the input image; the first network is configured to determine a defect degree of the industrial part in the input image based on the convolution feature of the input image; the second network is configured to perform feature dimension transformation on the convolution feature of the input image; the third network is configured to extract a convolution feature of a reflection image of the input image; and the fourth network is configured to determine a defect detection result of the industrial part in the input image based on the outputs of the first network, the second network and the third network.

[0192] A training unit 1002 is configured to train the first model based on multiple first training samples until a set convergence condition is reached, wherein the model parameters of the second model remain unchanged during the training process.

[0193] In an embodiment, the training unit 1002 is further configured to:

[0194] Call the second model to process multiple first input images, to obtain a defect detection result of each first input image.

[0195] Construct the multiple first training samples based on multiple second input images, wherein the second input images represent first input images with defects.

[0196] In an embodiment, the training unit 1002 is further configured to:

[0197] performing a sliding window processing in each second input image, and counting a proportion of first pixels in each sliding window covered area in the second input image, the first pixels representing pixels labeled as defects in the second input image;

[0198] constructing a first output matrix based on the proportion of the first pixels in each sliding window covered area; wherein,

[0199] the first training sample represents the second input image, and a label corresponding to the first training sample represents the first output matrix.

[0200] In an embodiment, the training unit 1002 is further configured to:

[0201] when training the first model based on a plurality of first training samples, calling a first loss function to process a second output matrix corresponding to each positive sample in the plurality of first training samples and a first output matrix corresponding to the positive sample, to obtain a first loss value, and calling the first loss function to process a second output matrix corresponding to each negative sample in the plurality of first training samples and a first output matrix corresponding to the negative sample, to obtain a second loss value; the second output matrix represents an output matrix obtained by processing the first training sample by calling the first model;

[0202] performing a summation calculation on the second loss value and a first product to obtain the third loss value; the first product represents a product of a first weight and the first loss value; the first weight represents a ratio between a number of negative samples and a number of positive samples in the plurality of first training samples, and the third loss value is used to determine whether the set convergence condition is reached.

[0203] In an embodiment, the training unit 1002 is further configured to:

[0204] training the defect detection model based on a plurality of second training samples to obtain the second model; wherein,

[0205] the second training sample represents an image of an industrial part without defects.

[0206] In an embodiment, the second model includes a teacher network encoder and a student network decoder running based on a reverse distillation mechanism, and correspondingly,

[0207] the first network is configured to output a first feature matrix representing a defect degree of the industrial part in the input image based on a feature distance between first convolutional features output by the teacher network encoder and second convolutional features output by the student network decoder;

[0208] The second network is configured to merge the first convolutional features of each layer output by the teacher network encoder, and perform dimension reduction processing on the merged third convolutional features.

[0209] In an embodiment, the third network is further configured to obtain the reflection image based on a Gaussian convolution calculation on the input image.

[0210] In actual application, the construction unit 1001 and the training unit 1002 can be implemented by a processor in a model training device.

[0211] It should be noted that the model training device provided in the above embodiments is only used as an example for the division of the above program modules in the model training process. In actual application, the above processing can be completed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the model training device and the model training method provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0212] Based on the above embodiments, the embodiments of the present application further provide an image detection device, which is described below with reference to Figure 11 The image detection device includes:

[0213] The first calling unit 1101 is configured to call a third model to process a third input image to obtain a third output matrix, the third output matrix representing a defect detection result of an industrial part contained in image content of the third input image, and the third input image representing an image containing the image content of the industrial part.

[0214] The third model represents a first model trained based on any of the above model training methods.

[0215] In an embodiment, the first calling unit 1101 is further configured to:

[0216] The first calling unit 1101 is further configured to:

[0217] The first calling unit 1101 is further configured to:

[0218] The first calling unit 1101 is further configured to:

[0219] In actual application, the first calling unit 1101 can be implemented by a processor in an image detection device.

[0220] It should be noted that the image detection device provided in the above embodiment is only used as an example to illustrate the division of the above program modules during image detection. In actual application, the above processing can be completed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the image detection device and the image detection method provided in the above embodiment belong to the same concept, and the specific implementation process is described in the method embodiment, which will not be repeated here.

[0221] Based on the hardware implementation of the above program modules, and in order to realize the model training method of the embodiment of the application, the application further provides an electronic device, which is described below with reference to Figure 12 The electronic device comprises:

[0222] The first communication interface 1201 is capable of information interaction with other devices.

[0223] The first processor 1202 is connected with the first communication interface 1201 to realize information interaction with other devices, and is used to run a computer program to execute the model training method provided in one or more technical solutions in the above embodiments. The computer program is stored on the first memory 1203.

[0224] Specifically, the first processor 1202 is configured to construct a first model, wherein the first model is obtained by adding a first network, a second network, a third network and a fourth network to a second model; the second model represents a defect detection model that has been trained and is used to detect defects of an industrial part in an input image based on convolution features of the input image; the first network is used to determine a defect degree of the industrial part in the input image based on the convolution features of the input image; the second network is used to perform feature dimension transformation on the convolution features of the input image; the third network is used to extract convolution features of a reflection image of the input image; and the fourth network is used to determine a defect detection result of the industrial part in the input image based on outputs of the first network, the second network and the third network.

[0225] The first model is trained based on a plurality of first training samples until a set convergence condition is reached; wherein the model parameters of the second model remain unchanged during the training process.

[0226] In an embodiment, the first processor 1202 is further configured to:

[0227] Call the second model to process a plurality of first input images to obtain a defect detection result of each first input image.

[0228] constructing the plurality of first training samples based on a plurality of second input images, the second input images representing first input images with corresponding defect detection results being defects.

[0229] In an embodiment, the first processor 1202 is further configured to:

[0230] performing a sliding window processing in each second input image and counting a proportion of first pixels in each sliding window coverage area in the second input image, the first pixels representing pixels labeled as defects in the second input image;

[0231] constructing a first output matrix based on the proportion of first pixels in each sliding window coverage area; wherein,

[0232] the first training sample represents the second input image, and a label corresponding to the first training sample represents the first output matrix.

[0233] In an embodiment, the first processor 1202 is further configured to:

[0234] when training the first model based on the plurality of first training samples, calling a first loss function to process a second output matrix corresponding to each positive sample in the plurality of first training samples and a first output matrix corresponding to the positive sample to obtain a first loss value, and calling the first loss function to process the second output matrix corresponding to each negative sample in the plurality of first training samples and the first output matrix corresponding to the negative sample to obtain a second loss value; the second output matrix represents an output matrix obtained by processing the first training sample by calling the first model;

[0235] summing the second loss value and a first product to obtain the third loss value; the first product represents a product of a first weight and the first loss value; the first weight represents a ratio between a number of negative samples and a number of positive samples in the plurality of first training samples, and the third loss value is used to determine whether the set convergence condition is reached.

[0236] In an embodiment, the first processor 1202 is further configured to:

[0237] training the defect detection model based on a plurality of second training samples to obtain the second model; wherein,

[0238] the second training sample represents an image of an industrial part without defects.

[0239] In an embodiment, the second model includes a teacher network encoder and a student network decoder running based on a reverse distillation mechanism, and correspondingly,

[0240] The first network is configured to output a first feature matrix based on a feature distance between the first convolutional features output by the teacher network encoder and the second convolutional features output by the student network decoder, the first feature matrix representing a defect degree of the industrial part in the input image.

[0241] The second network is configured to merge the first convolutional features of each layer output by the teacher network encoder, and perform dimension reduction processing on the third convolutional features obtained by the merging.

[0242] In an embodiment, the third network is further configured to obtain the reflection image based on a Gaussian convolution calculation on the input image.

[0243] It should be noted that the specific processing process of the first communication interface 1201 can be understood with reference to the above method.

[0244] Of course, in actual application, various components in the electronic device are coupled together through the bus system 1204. It can be understood that the bus system 1204 is used to realize the connection and communication between these components. In addition to including a data bus, the bus system 1204 also includes a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 1204 in the Figure 12 .

[0245] The first memory 1203 in the embodiment of the present application is used to store various types of data to support the operation in the electronic device. Examples of these data include: any computer programs used for operation on the electronic device.

[0246] The method disclosed in the above embodiments of the present application can be applied to or implemented by the first processor 1202. The first processor 1202 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in software form of the first processor 1202. The first processor 1202 mentioned above can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The first processor 1202 can implement or execute the disclosed methods, steps, and logic block diagrams in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of the present application, the hardware decoding processor can be directly embodied to execute the completion, or a combination of hardware and software modules in the decoding processor can be executed to complete. The software module can be located in a storage medium, which is located in the first memory 1203, and the first processor 1202 reads the information in the first memory 1203, and combines the hardware to complete the steps of the above method.

[0247] In exemplary embodiments, the electronic device can be implemented with one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, Microprocessors, or other electronic elements for performing the aforementioned methods.

[0248] It can be understood that the first memory 1203 of the embodiments of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM). The magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), sync link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory described in the embodiments of the present application is intended to include, but not limited to, these and any other suitable types of memory.

[0249] Based on the hardware implementation of the above program modules, and in order to implement the image detection method of the embodiments of the present application, the present application also provides an electronic device, which is described below with reference to Figure 13 The electronic device includes:

[0250] The second communication interface 1301 is capable of information interaction with other devices;

[0251] The second processor 1302 is connected with the second communication interface 1301 to realize information interaction with other devices, and is used to run a computer program to execute the image detection method provided by one or more technical solutions in the above embodiments. The computer program is stored on the second memory 1303.

[0252] Specifically, the second processor 1302 is configured to call a third model to process a third input image to obtain a third output matrix, the third output matrix representing a defect detection result of an industrial part contained in image content of the third input image, and the third input image representing an image containing the image content of the industrial part.

[0253] The third model represents a first model trained based on any of the above model training methods.

[0254] In an embodiment, the second processor 1302 is further configured to:

[0255] Binary conversion is performed on each element in the third output matrix to obtain a fourth output matrix;

[0256] Nearest neighbor interpolation calculation is performed on the fourth output matrix to obtain a first output image;

[0257] Connected component calculation is performed on the first output image to obtain a positioning frame, and the positioning frame is used to frame the image content corresponding to the defect of the industrial part in the third input image.

[0258] It should be noted that the specific processing process of the second communication interface 1301 can be understood with reference to the above method.

[0259] Of course, in actual application, various components in the electronic device are coupled together through the bus system 1304. It can be understood that the bus system 1304 is used to realize the connection and communication between the components. In addition to the data bus, the bus system 1304 also includes a power bus, a control bus and a status signal bus. However, in order to clearly illustrate the present application, all kinds of buses are marked as the bus system 1304 in the Figure 13

[0260] ​The second memory 1303 in the embodiments of the present application is configured to store various types of data to support operations in the electronic device. Examples of the data include any computer programs for operating on the electronic device.

[0261] The method disclosed in the embodiments of the present application can be applied to or implemented by the second processor 1302. The second processor 1302 can be an integrated circuit chip with processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in software form of the second processor 1302. The second processor 1302 can be a general processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The second processor 1302 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of the present application, the hardware decoding processor can be directly implemented or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in the storage medium, which is located in the second memory 1303, and the second processor 1302 reads the information in the second memory 1303 to complete the steps of the above method in combination with the hardware.

[0262] In the exemplary embodiments, the electronic device can be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, Microprocessors, or other electronic elements for executing the above method.

[0263] It can be understood that the second memory 1303 of the embodiments of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM). The magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), sync link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory described in the embodiments of the present application is intended to include, but not limited to, these and any other suitable types of memory.

[0264] In the example embodiments, the embodiments of the present application also provide a storage medium, specifically a computer readable storage medium, for example, including a first memory 1203 storing a computer program, which can be executed by the first processor 1202 to complete the steps of the aforementioned model training method. For another example, including a second memory 1303 storing a computer program, which can be executed by the second processor 1302 to complete the steps of the aforementioned image detection method.

[0265] In the example embodiments, the embodiments of the present application also provide a computer program product, including a computer program, which can be executed by the first processor 1202 to complete the steps of the aforementioned model training method. For another example, the computer program can be executed by the second processor 1302 to complete the steps of the aforementioned image detection method.

[0266] It should be noted that "first", "second", and the like are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence.

[0267] The term "and / or" herein is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. In addition, the term "one or more" herein means any combination of at least two of any one or more of a plurality, for example, including at least one of A, B, and C, which can represent including any one or more elements selected from the set consisting of A, B, and C.

[0268] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0269] The above is only a preferred embodiment of the present application, and is not intended to limit the protection scope of the present application.

Claims

1. A model training method, characterized in that, The method comprises: constructing a first model; the first model is obtained by adding a first network, a second network, a third network and a fourth network on a second model; the second model represents a defect detection model that has completed training and is used for defect detection of an industrial part in an input image based on a convolution feature of the input image; the first network is used for determining a defect degree of the industrial part in the input image based on the convolution feature of the input image; the second network is used for performing feature dimension transformation on the convolution feature of the input image; the third network is used for extracting a convolution feature of a reflection image of the input image; the fourth network is used for determining a defect detection result of the industrial part in the input image based on outputs of the first network, the second network and the third network; training the first model based on a plurality of first training samples until a set convergence condition is reached; wherein the model parameters of the second model remain unchanged during the training process.

2. The model training method of claim 1, wherein, The method further comprises: calling the second model to process a plurality of first input images to obtain a defect detection result of each first input image; constructing the plurality of first training samples based on a plurality of second input images, wherein the corresponding defect detection result of the second input image is a first input image with defects.

3. The model training method of claim 2, wherein, The method further comprises: performing sliding window processing in each second input image and counting the proportion of first pixels in each sliding window coverage area in the second input image, wherein the first pixels represent pixels labeled as defects in the second input image; constructing a first output matrix based on the proportion of first pixels in each sliding window coverage area; wherein, the first training sample represents the second input image, and the label corresponding to the first training sample represents the first output matrix.

4. The model training method of claim 3, wherein, The method further comprises: when training the first model based on a plurality of first training samples, calling a first loss function to process the second output matrix corresponding to each positive sample in the plurality of first training samples and the first output matrix corresponding to the positive sample to obtain a first loss value, and calling the first loss function to process the second output matrix corresponding to each negative sample in the plurality of first training samples and the first output matrix corresponding to the negative sample to obtain a second loss value; the second output matrix represents an output matrix obtained by calling the first model to process the first training sample; summing the second loss value and a first product to obtain the third loss value; the first product represents the product of a first weight and the first loss value; the first weight represents the ratio between the number of negative samples and the number of positive samples in the plurality of first training samples, and the third loss value is used to determine whether the set convergence condition is reached.

5. The model training method of claim 1, wherein, The method further comprises: training the defect detection model based on a plurality of second training samples to obtain the second model; wherein, the second training sample represents an image of an industrial part without defects. 6.The model training method of any one of claims 1-5, wherein, The second model comprises a teacher network encoder and a student network decoder running based on a reverse distillation mechanism, and correspondingly, The first network is configured to output a first feature matrix based on a feature distance between the first convolutional features output by the teacher network encoder and the second convolutional features output by the student network decoder, the first feature matrix representing a defect degree of the industrial part in the input image. The second network is configured to merge the first convolutional features of each layer output by the teacher network encoder, and perform dimension reduction processing on the third convolutional features obtained by the merging. 7.The model training method of any one of claims 1 to 5, characterized in that, The third network is further configured to obtain the reflection image based on Gaussian convolution calculation on the input image.

8. An image detection method characterized by, The method comprises: calling a third model to process a third input image to obtain a third output matrix, the third output matrix representing a defect detection result of an industrial part contained in image content of the third input image, the third input image representing an image containing image content of the industrial part; wherein the third model represents a first model trained based on the model training method according to any one of claims 1 to 7.

9. The image detection method of claim 8, wherein, The method further comprises: performing binary conversion on each element in the third output matrix to obtain a fourth output matrix; performing adjacent interpolation calculation on the fourth output matrix to obtain a first output image; performing connected component calculation on the first output image to obtain a positioning box; the positioning box is used to frame image content corresponding to the defect of the industrial part in the third input image.

10. A model training apparatus, comprising: comprises: a construction unit configured to construct a first model; the first model is obtained by adding a first network, a second network, a third network and a fourth network to a second model; the second model represents a defect detection model that has been trained and is used to detect defects of an industrial part in an input image based on convolutional features of the input image; the first network is configured to determine a defect degree of the industrial part in the input image based on the convolutional features of the input image; the second network is configured to perform feature dimension transformation on the convolutional features of the input image; the third network is configured to extract convolutional features of a reflection image of the input image; the fourth network is configured to determine a defect detection result of the industrial part in the input image based on outputs of the first network, the second network and the third network; a training unit configured to train the first model based on a plurality of first training samples until a set convergence condition is reached; wherein the model parameters of the second model remain unchanged during the training process.

11. An image detection apparatus characterized by comprising: comprises: a first calling unit configured to call a third model to process a third input image to obtain a third output matrix, the third output matrix representing a defect detection result of an industrial part contained in image content of the third input image, the third input image representing an image containing image content of the industrial part; wherein the third model represents a first model trained based on the model training method according to any one of claims 1 to 7.

12. An electronic device, comprising: comprises a first processor and a first communication interface; The first processor is configured to construct a first model, wherein the first model is obtained by adding a first network, a second network, a third network and a fourth network to a second model; the second model represents a defect detection model which has been trained and is used to detect defects of an industrial part in an input image based on a convolution feature of the input image; the first network is used to determine a defect degree of the industrial part in the input image based on the convolution feature of the input image; the second network is used to perform feature dimension transformation on the convolution feature of the input image; the third network is used to extract a convolution feature of a reflection image of the input image; and the fourth network is used to determine a defect detection result of the industrial part in the input image based on outputs of the first network, the second network and the third network. And, The first model is trained based on a plurality of first training samples until a set convergence condition is reached; wherein model parameters of the second model remain unchanged during the training process.

13. An electronic device, comprising: The second processor and a second communication interface are included. The second processor is configured to call a third model to process a third input image to obtain a third output matrix, wherein the third output matrix represents a defect detection result of an industrial part contained in image content of the third input image, and the third input image represents an image containing the image content of the industrial part. The third model represents the first model trained based on the model training method of any one of claims 1 to 7.

14. An electronic device, comprising: The first processor and a first memory for storing a computer program capable of running on the processor are included, The first processor is configured to execute the steps of the method of any one of claims 1 to 7 when running the computer program. The second processor and a second memory for storing a computer program capable of running on the processor are included, 15. An electronic device, comprising: The second processor is configured to execute the steps of the method of any one of claims 8 to 9 when running the computer program. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7, or to implement the steps of the method of any one of claims 8 to 9. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7, or to implement the steps of the method of any one of claims 8 to 9.

16. A storage medium having stored thereon a computer program, characterized in that ​ 17. A computer program product comprising a computer program, characterized in that, ​

Images