Image detection method and device, electronic equipment, storage medium and computer program product

Abstract

The invention provides an image detection method and device, electronic equipment, a storage medium and a computer program product, and relates to the technical field of image processing, and the method comprises the steps: obtaining a to-be-processed image; inputting the to-be-processed image into a preset detection model, and determining a detection result of the to-be-processed image; wherein the preset detection model is obtained by using a depth detection model to guide an initial detection model to train based on the sample data set; the sample data set comprises contrast-enhanced sample data; the parameter number of the depth detection model is greater than that of the initial detection model. Through the scheme in the embodiment of the invention, the detection accuracy of the to-be-processed image can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image detection method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0002] With increasing demands for industrial product quality, industrial defect detection has become increasingly important. Traditional defect detection methods require significant manpower, and manual inspection is also susceptible to harsh working environments, resulting in inconsistent results. The development of image processing technology has led to the use of machine vision for defect detection, providing a new approach for industrial defect detection. Training industrial product surface defect detection models using deep neural networks has become a research topic. Industrial defect detection faces several challenges: obtaining defect datasets is difficult, and blurry defect images can hinder the training of high-accuracy detection models. Furthermore, edge-deployed industrial defect detection models often suffer from small model sizes, which can lead to decreased detection accuracy. Summary of the Invention

[0003] The embodiments of this application provide an image detection method, apparatus, electronic device, storage medium, and computer program product that can improve the accuracy of image detection.

[0004] The technical solution of this application is implemented as follows:

[0005] This application provides an image detection method, including:

[0006] Obtain the image to be processed;

[0007] The image to be processed is input into a preset detection model to determine the detection result of the image to be processed;

[0008] The preset detection model is obtained by training an initial detection model based on a sample dataset using a deep detection model; the sample dataset includes contrast-enhanced sample data; and the number of parameters in the deep detection model is greater than the number of parameters in the initial detection model.

[0009] In the above scheme, before inputting the image to be processed into a preset detection model to determine the detection result of the image to be processed, the method further includes:

[0010] The image data in the acquired image dataset is subjected to noise reduction and contrast enhancement processing to obtain the sample dataset.

[0011] In the above scheme, before inputting the image to be processed into a preset detection model to determine the detection result of the image to be processed, the method further includes:

[0012] The training of the deep detection model based on the sample data determines soft labels, and the training of the initial detection model based on the sample data determines hard labels; wherein, both the deep detection model and the initial detection model include a feature extraction network with high-frequency information enhancement.

[0013] The soft and hard labels are used together to guide the training of the initial detection model until a predetermined training condition is met, thus obtaining the preset detection model.

[0014] In the above scheme, the step of determining soft labels by training the deep detection model based on the sample data, and determining hard labels by training the initial detection model based on the sample data, includes:

[0015] The sample data is input into the depth detection model to obtain the first feature, and the first feature is enhanced with high-frequency information based on the first feature extraction network, and finally the soft label is output.

[0016] The sample data is input into the initial detection model to obtain the second feature, and the second feature is enhanced with high-frequency information based on the second feature extraction network to finally output the hard label.

[0017] In the above scheme, the step of enhancing the high-frequency information of the second feature based on the second feature extraction network and finally outputting the hard label includes:

[0018] The second feature is pooled to obtain low-frequency feature information, and high-frequency feature information is determined based on the difference between the second feature and the low-frequency feature information;

[0019] After extracting features from the high-frequency feature information and the low-frequency feature information respectively, they are fused to determine the fused feature;

[0020] The fused features are input into the next network until the hard label is output.

[0021] In the above scheme, the step of combining the soft labels and the hard labels to guide the initial detection model in training until a predetermined training condition is met, thereby obtaining the preset detection model, includes:

[0022] A first loss for the depth detection model is determined based on the difference between the soft label and the hard label, and a second loss for the initial detection model is determined based on the difference between the hard label and the true result; wherein the true result is the true result corresponding to the sample data;

[0023] A third loss is determined for the initial detection model based on the first loss and the second loss;

[0024] The model parameters of the initial detection model are updated using gradients based on the third loss, and the initial detection model is trained using the hard label and soft label corresponding to the next sample data until a predetermined training condition is met, thus obtaining the preset detection model.

[0025] This application also provides an image detection device, including:

[0026] The acquisition unit is used to acquire the image to be processed;

[0027] An image processing unit is used to input the image to be processed into a preset detection model and determine the detection result of the image to be processed.

[0028] The preset detection model is obtained by training an initial detection model based on a sample dataset using a deep detection model; the sample dataset includes contrast-enhanced sample data; and the number of parameters in the deep detection model is greater than the number of parameters in the initial detection model.

[0029] This application also provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the steps in the above-described method.

[0030] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the above-described method.

[0031] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described method.

[0032] In this embodiment, an image to be processed is acquired; the image to be processed is input into a preset detection model to determine the detection result of the image to be processed; wherein, the preset detection model is obtained by training an initial detection model based on a sample dataset using a deep detection model; the sample dataset includes contrast-enhanced sample data; the number of parameters in the deep detection model is greater than the number of parameters in the initial detection model. Since the sample data in this embodiment is contrast-enhanced data, the problem of image blurring in training data in related technologies is solved, and the training effect of the initial detection model can be improved by using sample data with obvious contrast. Furthermore, in this embodiment, the initial detection model is trained by a deep detection model with a large number of parameters, which means that the initial detection model with a small number of parameters can also be affected by a large number of parameters during training, resulting in more accurate gradient updates. This makes the performance of the trained preset detection model better, thereby improving the detection accuracy of the image to be processed. Attached Figure Description

[0033] Figure 1 A schematic diagram of an optional process for the image detection method provided in an embodiment of this application;

[0034] Figure 2 A schematic diagram of an optional process for the image detection method provided in an embodiment of this application;

[0035] Figure 3 This is an optional effect diagram of the image detection method provided in the embodiments of this application;

[0036] Figure 4 A schematic diagram of an optional process for the image detection method provided in an embodiment of this application;

[0037] Figure 5 A schematic diagram of an optional process for the image detection method provided in an embodiment of this application;

[0038] Figure 6 A schematic diagram of an optional process for the image detection method provided in an embodiment of this application;

[0039] Figure 7 This is a schematic diagram of the structure of the image detection device provided in the embodiments of this application;

[0040] Figure 8 This is a schematic diagram of a hardware entity of an electronic device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0045] This application provides an image detection method. Please refer to [link / reference]. Figure 1 This is an optional flowchart illustrating the image detection method provided in the embodiments of this application, which will be combined with... Figure 1 The steps shown are explained below:

[0046] S101. Obtain the image to be processed.

[0047] In this embodiment of the application, the image detection device acquires the image to be detected.

[0048] The image inspection device can be a detection terminal in an industrial surface defect detection scenario or in other image inspection scenarios.

[0049] In this embodiment of the application, after obtaining the image to be processed, noise reduction and contrast enhancement can also be performed on the image to be processed.

[0050] S102. Input the image to be processed into a preset detection model to determine the detection result of the image to be processed; wherein, the preset detection model is obtained by training an initial detection model based on a sample dataset using a depth detection model; the sample dataset includes contrast-enhanced sample data; the number of parameters of the depth detection model is greater than the number of parameters of the initial detection model.

[0051] In this embodiment of the application, the image detection device inputs the acquired image to be processed into a preset detection model. By processing the image to be processed through the preset detection model, the detection result of whether the image to be processed has defects can be determined.

[0052] Before performing detection and processing on the images to be processed, a preset detection model needs to be trained. This training process can be performed by an image detection device or by other servers or terminals with corresponding functions. After training, the preset detection model is deployed on the image detection device. In this embodiment, samples from the sample dataset are used to input a deep detection model and an initial detection model. The soft labels output by the deep detection model are used as guidance, combined with the hard labels output by the initial detection model to determine the loss. The loss is then used to update the gradient of the initial detection model until a predetermined training condition is met, resulting in the preset detection model. The number of parameters in the deep detection model is greater than the number of parameters in the initial detection model. For example, the deep detection model can be a YOLOv5s (You Only Look Once version 5, smallest) detection model, and the initial detection model can be a YOLOv5n (You Only Look Once version 5, nano) detection model. The sample dataset includes contrast-enhanced sample data.

[0053] This application provides an image detection method. After randomly cropping and augmenting the collected industrial defect data, contrast enhancement is performed on the dataset images using contrast-limited adaptive histogram equalization. The high-frequency enhancement module is replaced in the industrial defect detection network to increase the network's focus on high-frequency information. The contrast-enhanced images and the original images are used together as a dataset for training the industrial defect detection network. During training, a large network (deep detection model) is first selected for industrial defect detection training to obtain the network model. Then, a smaller network model (initial detection model) is selected. Based on a knowledge distillation strategy, the training of the smaller network model is guided by the larger network model, ultimately resulting in an industrial defect detection model (preset detection model) with low computational cost and few parameters.

[0054] In this embodiment, an image to be processed is acquired; the image to be processed is input into a preset detection model to determine the detection result of the image to be processed; wherein, the preset detection model is obtained by training an initial detection model based on a sample dataset using a deep detection model; the sample dataset includes contrast-enhanced sample data; the number of parameters in the deep detection model is greater than the number of parameters in the initial detection model. Since the sample data in this embodiment is contrast-enhanced data, the problem of image blurring in training data in related technologies is solved. Using sample data with obvious contrast can improve the training effect of the initial detection model and enhance the model's performance. Furthermore, in this embodiment, the initial detection model is trained using a deep detection model with a larger number of parameters, resulting in the initial detection model with a smaller number of parameters being influenced by a large number of parameters during training, leading to more accurate gradient updates. This results in better performance of the trained preset detection model, thereby improving the detection accuracy of the image to be processed.

[0055] Please see Figure 2 The following is an optional flowchart of the image detection method provided in the embodiments of this application, which will be described in conjunction with the steps:

[0056] S201. Perform noise reduction and contrast enhancement processing on the image data in the acquired image dataset to obtain the sample dataset.

[0057] In this embodiment of the application, in the scenario of detecting industrial defect images, there is a problem of low contrast between the defect target and the background and a lot of noise, which affects the quality of model training. To address this problem, after acquiring an image dataset including industrial defects, the image processing device can use a contrast-limited histogram equalization algorithm to perform contrast enhancement and noise reduction processing on each image data, thereby obtaining a sample dataset.

[0058] In this embodiment, during the noise reduction and contrast enhancement processing of image data, the input image data is divided into non-overlapping sub-blocks of equal size, and histogram equalization is performed on each sub-block. The histogram of each sub-block is calculated, and the cumulative distribution function is calculated based on the histogram. Before calculating the cumulative distribution function, the histogram is cropped to limit its maximum and minimum values, preventing excessive skewness. The purpose is to limit contrast and prevent over-enhancement of noise. Especially when noise is present in the image, limiting the range of the histogram can reduce the amplification effect of noise. Then, the cumulative distribution function is used to transform the pixel values ​​in the sub-blocks to achieve equalization, ultimately obtaining sample data to form a sample dataset. To smoothly transition the processing results of each sub-block to the entire image, this scheme uses bilinear interpolation to stitch the blocks together.

[0059] For example, in combination Figure 3The three images in the image dataset have relatively weak contrast. After contrast enhancement processing, a contrast-enhanced sample dataset is obtained. The defect features are more obvious in the contrast-enhanced images. The sample data in the dataset can be re-divided into training, validation, and test sets for model training.

[0060] S202. Soft labels are determined by training the deep detection model based on the sample data, and hard labels are determined by training the initial detection model based on the sample data; wherein, both the deep detection model and the initial detection model include a feature extraction network with high-frequency information enhancement.

[0061] In this embodiment, the image processing device can replace the C3 convolutional module in the deep detection model with a feature extraction network that has high-frequency information enhancement characteristics, and replace the C3 convolutional module in the initial detection model with a feature extraction network that has high-frequency information enhancement characteristics. The same sample data from the sample dataset can be input into both the deep detection model and the initial detection model. The initial detection model processes the sample data to obtain hard labels for the detection results, while the deep detection model processes the sample data to obtain soft labels for the detection results.

[0062] S203. The initial detection model is trained by combining the soft label and the hard label until the predetermined training conditions are met, and then the preset detection model is obtained.

[0063] In this embodiment, the image processing device can determine the loss for the initial detection model by combining the differences between the soft label and the hard label and the true result, and perform gradient updates on the initial detection model based on the loss until the predetermined training conditions are met, thereby stopping the training and obtaining the preset detection model.

[0064] In this embodiment, the image processing device can determine a first loss for the depth detection model based on the difference between the soft label and the true result, and determine a second loss for the initial detection model based on the difference between the hard label and the true result. Based on the first and second losses, a final third loss for the initial detection model is determined. The initial detection model is then updated using gradients based on the third loss until a predetermined training condition is met, at which point training stops, resulting in a preset detection model.

[0065] In this embodiment, noise reduction and contrast enhancement are performed on the image data in the acquired image dataset to obtain a sample dataset. Soft labels are determined by training a deep detection model based on the sample data, and hard labels are determined by training an initial detection model based on the sample data. Both the deep detection model and the initial detection model include a feature extraction network with enhanced high-frequency information. The soft and hard labels are used together to guide the training of the initial detection model until a predetermined training condition is met, resulting in a preset detection model. Since the sample data is contrast-enhanced, the problem of image blurring in training data in related technologies is solved. Using sample data with significant contrast can improve the training effect of the initial detection model and enhance its performance. Furthermore, in this embodiment, the soft labels of the deep detection model with a large number of parameters guide the training of the initial detection model. This allows the initial detection model with a smaller number of parameters to be influenced by a large number of parameters during training, resulting in more accurate gradient updates. This leads to better performance of the trained preset detection model and improves the detection accuracy of the processed image. In addition, since both the initial detection model and the deep detection model include a high-frequency information-enhanced feature extraction network, the high-frequency information-enhanced feature extraction network can learn more image details, edge information, etc. from the image data. The more content learned, the better the performance of the trained model. Therefore, the preset detection model has higher detection accuracy for the image data to be processed.

[0066] Please see Figure 4 This is a schematic diagram of an optional process for the image detection method provided in an embodiment of this application. Figure 2 S202 shown can also be implemented via S301 to S302, which will be explained in conjunction with the steps:

[0067] S301. Input the sample data into the depth detection model to obtain the first feature, and perform high-frequency information enhancement on the first feature based on the first feature extraction network, and finally output the soft label.

[0068] In this embodiment, the image detection device can input sample data into a depth detection model, obtain a first feature through the pre-processing network of the depth detection model, transmit the first feature to a first feature extraction network for high-frequency information enhancement, extract high-frequency and low-frequency features from the first feature using the first feature extraction network and fuse them to obtain a fused feature, which is then transmitted to a subsequent network for processing, and finally outputs a detection result soft label.

[0069] In this embodiment, during image processing, the model learns more information about image details and edges through high-frequency feature maps, which helps in saliency detection. Low-frequency feature maps contain less information. Therefore, this invention uses a plug-and-play high-frequency enhancement module to replace the C3 convolutional module in the YOLOv5s industrial defect detection network, which can improve the training effect of the model.

[0070] Soft labels, compared to hard labels, provide more flexible and richer information. In classification problems, a soft label can be a probability distribution representing the likelihood of a sample belonging to each category. This approach helps the model learn more detailed data features and alleviates overfitting to some extent. For example, in image classification, an image may contain features from multiple categories simultaneously; soft labels can more accurately reflect this complexity.

[0071] S302. Input the sample data into the initial detection model to obtain the second feature, and enhance the second feature with high-frequency information based on the second feature extraction network, and finally output the hard label.

[0072] In this embodiment, the image detection device can input sample data into an initial detection model, obtain a second feature through the pre-processing network of the initial detection model, transmit the second feature to a second feature extraction network for high-frequency information enhancement, extract high-frequency and low-frequency features from the second feature using the second feature extraction network and fuse them to obtain a fused feature that is transmitted to a subsequent network for processing, and finally output the detection result hard label.

[0073] Hard labels, in the traditional sense, assign a single, definitive category or value to each sample, such as class labels (0, 1, 2, ...) in classification problems. While hard labels provide clear classification boundaries, this absolute approach may overlook subtle differences and uncertainties among the data in certain situations.

[0074] In this embodiment, sample data is input into a deep detection model to obtain a first feature, and high-frequency information enhancement is performed on the first feature based on a first feature extraction network, ultimately outputting a soft label. Sample data is input into an initial detection model to obtain a second feature, and high-frequency information enhancement is performed on the second feature based on a second feature extraction network, ultimately outputting a hard label. Because both the deep detection model and the initial detection model contain high-frequency information enhancement networks, and due to the characteristics of high-frequency information enhancement networks, both the deep detection model and the initial detection model can learn more information about image details, edges, etc., which helps in saliency detection and thus improves the performance of the preset detection model obtained after training.

[0075] The illustrated S301 can also be implemented via S3011 to S3013, which will be explained in conjunction with the steps:

[0076] S3011. The first feature is pooled to obtain low-frequency feature information, and high-frequency feature information is determined based on the difference between the first feature and the low-frequency feature information.

[0077] In this embodiment of the application, the first feature extraction network of the depth detection model in the image processing device performs pooling processing on the first feature to obtain low-frequency feature information, and determines high-frequency feature information based on the difference between the first feature and the low-frequency feature information.

[0078] S3012. After extracting the features of the high-frequency feature information and the low-frequency feature information respectively, they are fused to determine the fused features.

[0079] In this embodiment, the first feature extraction network of the depth detection model of the image processing device extracts features from high-frequency feature information and low-frequency feature information respectively, and fuses the extracted features according to channels to obtain fused features.

[0080] S3013. Input the fused features into the next network until the soft label is output.

[0081] In this embodiment, the depth detection model of the image processing device transmits the fused features to the next network of the first feature extraction network for processing until a soft label for the sample data detection is output.

[0082] The shown S302 can also be implemented by S3021 to S3023, which will be explained in conjunction with the steps:

[0083] S3021. The second feature is pooled to obtain low-frequency feature information, and high-frequency feature information is determined based on the difference between the second feature and the low-frequency feature information.

[0084] In this embodiment, the second feature extraction network of the initial detection model of the image processing device utilizes pooling layers to separate high-frequency and low-frequency feature information in the second feature. Low-frequency feature information FM low As shown in formula (1), it is obtained from the second feature FM through a max pooling layer. High-frequency feature information FM high Formula (2) is obtained by subtracting the low-frequency feature map from the second feature.

[0085] FM low =MaxPooling(FM) Formula (1)

[0086] FM high =FM - FM low Formula (2)

[0087] S3022. After extracting the features of the high-frequency feature information and the low-frequency feature information respectively, they are fused to determine the fused features.

[0088] In this embodiment, the second feature extraction network of the initial detection model of the image processing device can use the convolutional layer and the Bottleneck network therein to extract high-frequency feature information and low-frequency feature information, and fuse the two extracted features to determine the fused feature.

[0089] In this embodiment, high-frequency feature information and low-frequency feature information are obtained. The second feature extraction network processes the high-frequency feature information and low-frequency feature information using convolution and Bottleneck, making full use of the high-frequency feature information. Finally, the low-frequency features and high-frequency features are fused through convolutional blocks. Among them, Bottleneck is calculated as shown in formula (3), where x represents the input feature information and Conv represents the convolutional layer.

[0090] Bottleneck = x + Conv(Conv(x)) Formula (3)

[0091] The high-frequency feature extraction process is shown in Equation (4), which involves passing through a convolutional layer (Conv) followed by a Bottleneck module. The low-frequency feature extraction process is shown in Equation (5), which involves passing through a convolutional layer followed by two Bottleneck modules. The fusion process is shown in Equation (6), which involves concatenating the high-frequency and low-frequency feature branches by channel and then fusing the two branches using convolution to obtain the fused feature F. The fusion of high-frequency and low-frequency features can enhance the information flow and expressive power of the model.

[0092] F h =Bottleneck(Conv(FM) high )) Formula (4)

[0093] F l =Bottleneck(Bottleneck(Conv(FM low ))) Formula (5)

[0094] F = Conv(torch.cat(F) l ,F h )) Formula (6)

[0095] S3023. Input the fused features into the next network until the hard label is output.

[0096] In this embodiment, the second feature extraction network of the initial detection model of the image processing device transmits the fused features to the next subsequent network for processing, until the initial detection model initially detects the hard label of the sample data.

[0097] Because the initial detection model enhances the high-frequency information of the second feature, it can learn more image details, edges, and other information, which helps in saliency detection and thus improves the performance of the preset detection model obtained after training.

[0098] Please see Figure 5 This is a schematic diagram of an optional process for the image detection method provided in an embodiment of this application. Figure 2 S203 shown can also be implemented via S401 to S403, which will be explained in conjunction with the steps:

[0099] S401. Determine a first loss for the depth detection model based on the difference between the soft label and the hard label, and determine a second loss for the initial detection model based on the difference between the hard label and the true result.

[0100] In this embodiment, the output soft labels can be the first location detection result, the first target detection result, and the first category detection result of the depth detection model for the sample data. The output hard labels can be the second location detection result, the second target detection result, and the second category detection result of the depth detection model for the sample data. The obtained true results can be the location result, the target result, and the category result for the sample data. The image processing device can determine a first loss for the depth detection model based on the difference between the first location detection result and the second location detection result, the difference between the first target detection result and the second target detection result, and the difference between the first category detection result and the second category detection result. The image processing device can determine a second loss for the initial detection model based on the difference between the second location detection result and the location result, the difference between the second target detection result and the target result, and the difference between the second category detection result and the category result.

[0101] In this embodiment, due to resource constraints, deep neural network models are difficult to deploy on resource-limited devices. Therefore, industrial defect detection algorithms require a model with low computational cost and few parameters, while maintaining high detection accuracy. This scheme uses a knowledge distillation strategy to train the industrial defect detection model. First, a large teacher network (deep detection model) is trained, extracting knowledge from the larger deep detection model into a smaller network. This scheme uses YOLOv5s as the teacher network, and then uses the knowledge distillation strategy to train a small student network YOLOv5n (initial detection model), meeting the accuracy and computational resource requirements of industrial defect detection. The knowledge distillation strategy aims to make the teacher network and student network have similar prediction results, thus requiring increased correlation between them. This scheme uses both hard labels from the teacher network and soft labels from the student network to constrain the training of the student network. This scheme is an industrial defect detection model that uses the positional loss L between the hard labels of the teacher network and the soft labels of the student network. tbox Target loss L tobj Category loss L tcls The positional loss, target loss, and category loss between teacher network hard labels and student network soft labels are calculated using the mean squared error loss function, as shown in formula (7). The first loss L... d Defined as the positional loss L between the corresponding detection results of the teacher network and the student network respectively. tbox Target loss L tobj Category loss L tcls The weighted summation can be shown in formula (8).

[0102]

[0103] Where n is the number of samples in a batch of sample data, y i The detection results are for the soft tags. For the detection results corresponding to hard labels, the mean squared error loss is quantified by averaging the squares of the differences between the predicted and actual values.

[0104] L d =α·L tbox +β·L tobj +γ·L tobj Formula (8)

[0105] Where α is L tbox The corresponding weights, where β is the corresponding L tobj The weights, γ is the corresponding L tobj The weight.

[0106] S402. Determine a third loss for the initial detection model based on the first loss and the second loss.

[0107] In this embodiment of the application, the image detection device can determine the third loss by weighted summation of the first loss and the second loss.

[0108] In this embodiment of the application, the third loss L of the reconstructed industrial defect detection network is replaced by the first loss L. d With the second loss L s The combination of the distillation loss L of the depth detection model and the initial detection model. d With students' network losses L s A weighted summation is performed, as shown in formula (9), and the training of the initial detection model is guided by both teacher network soft labels and student network hard labels.

[0109] L = a·L d +b·L s Formula (9)

[0110] Where a is L d The corresponding weight, b is L s The corresponding weights are: a = 0.7 and b = 0.3.

[0111] S403. Based on the third loss, the model parameters of the initial detection model are updated by gradient, and the hard label and soft label corresponding to the next sample data are used to guide the initial detection model to train until the predetermined training conditions are met, and then the preset detection model is obtained.

[0112] In this embodiment of the application, the image processing device can perform gradient updates on the model parameters of the initial detection model based on the third loss, and use the hard label and soft label corresponding to the next sample data to guide the initial detection model to be trained until a predetermined training condition is reached, thereby obtaining the preset detection model.

[0113] In this embodiment, a first loss for the deep detection model is determined based on the difference between soft and hard labels, and a second loss for the initial detection model is determined based on the difference between the hard label and the true result. A third loss for the initial detection model is then determined based on the first and second losses. The model parameters of the initial detection model are updated using gradients based on the third loss, and the initial detection model is trained using the hard and soft labels corresponding to the next sample data until a predetermined training condition is met, resulting in a preset detection model. In this way, by using a deep detection model with a large number of parameters to guide the training of the initial detection model, even the initial detection model with a smaller number of parameters can be influenced by a large number of parameters during training, resulting in more accurate gradient updates. This leads to better performance of the trained preset detection model, thereby improving the detection accuracy of the image being processed.

[0114] This application proposes an image detection method for training an industrial defect detection model (preset detection model) that is computationally inexpensive, easy to deploy, and highly accurate. The overall flowchart of this invention is as follows: Figure 6 As shown, an industrial defect dataset is first constructed using collected industrial image data. Next, the dataset undergoes cropping and preprocessing. Then, data augmentation operations such as random cropping, random flipping, random rotation, and random scaling are performed on the preprocessed data to expand the dataset. Simultaneously, histogram equalization with limited contrast is used for contrast enhancement, resulting in a defect dataset with higher contrast and more obvious defect features. Training, validation, and test sets are then created. A high-frequency augmentation module is added to the industrial defect detection network to obtain an augmented network for industrial defect detection training. During training, a large network, such as the YOLOv5s detection network, is trained first, followed by a smaller network, such as the YOLOv5n, using a knowledge distillation strategy to meet the accuracy and computational resource requirements of industrial defect detection.

[0115] Please see Figure 6 The following steps will be explained:

[0116] S11. Obtain the image dataset.

[0117] In this embodiment of the application, an image dataset of surface defects in industrial products is obtained.

[0118] S12. Determine the sample dataset.

[0119] In this embodiment of the application, the image dataset is cropped and preprocessed, and then the preprocessed data is augmented to expand the dataset. In step 1, the dataset is subjected to a limited contrast histogram equalization algorithm to obtain image data with enhanced contrast, and the dataset is divided into a training set, a validation set, and a test set to obtain a sample dataset.

[0120] S13, Deep detection model training.

[0121] In this embodiment, a deep detection model is selected, and an enhanced network is obtained by replacing the C3 network module with a high-frequency enhancement module. Sample data from the sample dataset is input into the deep detection model for training, and a soft label is output.

[0122] S14. Initial detection model training.

[0123] In this embodiment, an initial detection model is selected, and the model parameters of the initial detection model are smaller than those of the deep detection model. An augmented network is obtained by replacing the C3 network module with a high-frequency augmentation module. The same sample data is input into the initial detection model for training, and a hard label is output. A knowledge distillation strategy is applied to guide the initial detection model through a training iteration using both the soft and hard labels. Then, the initial detection model is trained using the next batch of sample data until a predetermined training condition is met, at which point the training stops, resulting in a preset detection model.

[0124] Please see Figure 7 This is a schematic diagram of the structure of the image detection device provided in the embodiments of this application.

[0125] This application also provides an image detection device 800, including: an acquisition unit 801 and an image processing unit 802.

[0126] The acquisition unit is used to acquire the image to be processed;

[0127] An image processing unit is used to input the acquired image to be processed into a preset detection model and determine the detection result of the image to be processed.

[0128] The preset detection model is obtained by training an initial detection model based on a sample dataset using a deep detection model; the sample dataset includes contrast-enhanced sample data; and the number of parameters in the deep detection model is greater than the number of parameters in the initial detection model.

[0129] In this embodiment of the application, the image processing unit 802 in the image detection device 800 is used to perform noise reduction and contrast enhancement processing on the image data in the acquired image dataset to obtain the sample dataset.

[0130] In this embodiment of the application, the image processing unit 802 in the image detection device 800 is used to determine soft labels by training the depth detection model based on the sample data, and to determine hard labels by training the initial detection model based on the sample data; wherein, both the depth detection model and the initial detection model include a feature extraction network with high frequency information enhancement.

[0131] The soft and hard labels are used together to guide the training of the initial detection model until a predetermined training condition is met, thus obtaining the preset detection model.

[0132] In this embodiment of the application, the image processing unit 802 in the image detection device 800 is used to input the sample data into the depth detection model to obtain the first feature, and to perform high-frequency information enhancement on the first feature based on the first feature extraction network, and finally output the soft label;

[0133] The sample data is input into the initial detection model to obtain the second feature, and the second feature is enhanced with high-frequency information based on the second feature extraction network to finally output the hard label.

[0134] In this embodiment of the application, the image processing unit 802 in the image detection device 800 is used to perform pooling processing on the second feature to obtain low-frequency feature information, and to determine high-frequency feature information based on the difference between the second feature and the low-frequency feature information;

[0135] After extracting features from the high-frequency feature information and the low-frequency feature information respectively, they are fused to determine the fused feature;

[0136] The fused features are input into the next network until the hard label is output.

[0137] In this embodiment of the application, the image processing unit 802 in the image detection device 800 is used to determine a first loss for the depth detection model based on the difference between the soft label and the hard label, and to determine a second loss for the initial detection model based on the difference between the hard label and the true result; wherein, the true result is the true result corresponding to the sample data;

[0138] A third loss is determined for the initial detection model based on the first loss and the second loss;

[0139] The model parameters of the initial detection model are updated using gradients based on the third loss, and the initial detection model is trained using the hard label and soft label corresponding to the next sample data until a predetermined training condition is met, thus obtaining the preset detection model.

[0140] It should be noted that, in the embodiments of this application, if the above-described item information processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an item information processing device (which may be a personal computer, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0141] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of a method on one side of an image processing apparatus.

[0142] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0143] It should be noted that, Figure 8 A schematic diagram of a hardware entity of an electronic device provided in an embodiment of this application, such as... Figure 8 As shown, this application embodiment provides an electronic device 900, including a memory 902 and a processor 901. The memory 902 stores a computer program that can run on the processor 901. When the processor 901 executes the program, it implements the steps in the above-described method, wherein;

[0144] Processor 901 typically controls the overall operation of electronic device 900.

[0145] The memory 902 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) in the processor 901 and various modules in the electronic device 900. It can be implemented by flash memory or random access memory (RAM).

[0146] Correspondingly, this application also provides a computer program product, including a computer program that can be executed by the processor 901 of the electronic device 900 to complete the steps in the method on the side of the image processing device.

[0147] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0148] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the apparatus or units can be electrical, mechanical, or other forms.

[0150] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0151] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0152] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0153] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0154] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image detection method, characterized in that, include: Obtain the image to be processed; The image to be processed is input into a preset detection model to determine the detection result of the image to be processed; The preset detection model is obtained by training an initial detection model based on a sample dataset using a deep detection model; the sample dataset includes contrast-enhanced sample data; and the number of parameters in the deep detection model is greater than the number of parameters in the initial detection model.

2. The image detection method according to claim 1, characterized in that, Before inputting the image to be processed into a preset detection model to determine the detection result of the image to be processed, the method further includes: The image data in the acquired image dataset is subjected to noise reduction and contrast enhancement processing to obtain the sample dataset.

3. The image detection method according to claim 1, characterized in that, Before inputting the image to be processed into a preset detection model to determine the detection result of the image to be processed, the method further includes: The training of the deep detection model based on the sample data determines soft labels, and the training of the initial detection model based on the sample data determines hard labels; wherein, both the deep detection model and the initial detection model include a feature extraction network with high-frequency information enhancement. The soft and hard labels are used together to guide the training of the initial detection model until a predetermined training condition is met, thus obtaining the preset detection model.

4. The image detection method according to claim 3, characterized in that, The process of determining soft labels by training the deep detection model based on the sample data, and determining hard labels by training the initial detection model based on the sample data, includes: The sample data is input into the depth detection model to obtain the first feature, and the first feature is enhanced with high-frequency information based on the first feature extraction network, and finally the soft label is output. The sample data is input into the initial detection model to obtain the second feature, and the second feature is enhanced with high-frequency information based on the second feature extraction network to finally output the hard label.

5. The image detection method according to claim 4, characterized in that, The process of enhancing the high-frequency information of the second feature based on the second feature extraction network, and finally outputting the hard label, includes: The second feature is pooled to obtain low-frequency feature information, and high-frequency feature information is determined based on the difference between the second feature and the low-frequency feature information; After extracting features from the high-frequency feature information and the low-frequency feature information respectively, they are fused to determine the fused feature; The fused features are input into the next network until the hard label is output.

6. The image detection method according to any one of claims 3 to 5, characterized in that, The process of training the initial detection model by combining the soft and hard labels until a predetermined training condition is met, thereby obtaining the preset detection model, includes: A first loss for the depth detection model is determined based on the difference between the soft label and the hard label, and a second loss for the initial detection model is determined based on the difference between the hard label and the true result; wherein the true result is the true result corresponding to the sample data; A third loss is determined for the initial detection model based on the first loss and the second loss; The model parameters of the initial detection model are updated using gradients based on the third loss, and the initial detection model is trained using the hard label and soft label corresponding to the next sample data until a predetermined training condition is met, thus obtaining the preset detection model.

7. An image detection device, characterized in that, include: The acquisition unit is used to acquire the image to be processed; An image processing unit is used to input the acquired image to be processed into a preset detection model and determine the detection result of the image to be processed. The preset detection model is obtained by training an initial detection model based on a sample dataset using a deep detection model; the sample dataset includes contrast-enhanced sample data; and the number of parameters in the deep detection model is greater than the number of parameters in the initial detection model.

8. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program that can run on the processor, the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images