Device defect detection method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a device defect detection method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring a device sample image; inputting the device sample image into a to-be-trained device defect detection model to obtain prediction categories and prediction probabilities corresponding to different moments output by the device defect detection model; determining a prediction change of the device defect detection model according to the prediction category and the prediction probability; updating a weight corresponding to the device sample image according to the prediction change; training a device defect detection model according to the device sample image with the updated weight; and when the device defect detection model satisfies a preset convergence condition, obtaining a trained target device defect detection model. According to the embodiment of the invention, under the condition that the number of the normal device sample images is different from that of the defective device sample images, relatively high weights can be dynamically allocated for a small number of categories with high learning difficulty, and the recognition performance of the device defect detection model on the small number of categories is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a device defect detection method, a device defect detection apparatus, an electronic device, and a computer-readable storage medium. Background Art

[0002] The air conditioning refrigeration system includes multiple components, among which the two devices (evaporator and condenser) in the air conditioning refrigeration system are important components of the air conditioning refrigeration system, and their production quality is particularly important in the production process of air conditioners. The automatic production of the two devices includes production processes such as inserting copper tubes, installing baffles, expanding tube mouths, drying, nitrogen filling, inserting elbows, and cutting. Among them, the copper tube insertion, baffle installation, and tube mouth expansion links are prone to defects such as missing copper tubes, missing baffles, and tube mouth fracturing. Therefore, it is necessary to perform molding quality inspection of the two devices before the drying process. The types of defects of the two devices in the air conditioning refrigeration system are varied, and it is difficult to detect them by artificial feature extraction methods.

[0003] The model based on convolutional deep learning technology can extract the desired features in the production process of air-conditioning and refrigeration systems in a data-driven manner, which can be used to complete the defect detection of the two devices. However, the training of the model requires a large amount of sample data, and the images of the two devices at the production site, especially the defective images, are difficult to collect, resulting in low classification accuracy of the model, which is difficult to meet the needs of automatic detection. Summary of the invention

[0004] The embodiments of the present invention provide a device defect detection method, apparatus, electronic device and computer-readable storage medium to solve the problem that device images at the production site, especially images of defective devices, are difficult to collect, resulting in low classification accuracy of the model and difficulty in meeting the needs of automatic detection.

[0005] An embodiment of the present invention discloses a device defect detection method, the method comprising:

[0006] Acquire device sample images; wherein the device sample images have corresponding actual categories and weights, the device sample images at least include normal device sample images and defective device sample images, the actual categories at least include normal categories and defective categories, and the number of the normal device sample images and the number of the defective device sample images are different;

[0007] Input the device sample image into the device defect detection model to be trained, and obtain the prediction category and prediction probability corresponding to the different moments output by the device defect detection model; the prediction probability is the probability that the device defect detection model predicts that the device image is the prediction category;

[0008] Determining a predicted change of the device defect detection model according to the predicted category and the predicted probability;

[0009] updating the weight corresponding to the device sample image according to the predicted change;

[0010] Training the device defect detection model according to the device sample images with updated weights;

[0011] When the device defect detection model meets the preset convergence condition, a trained target device defect detection model is obtained; the target device defect detection model is used to detect the category corresponding to the input device image.

[0012] In one embodiment of the present invention, the device at least includes an evaporator and a condenser on an air-conditioning refrigeration system.

[0013] In an embodiment of the present invention, the predicted change includes a forgetting learning predicted change of the device defect detection model in a forgetting learning direction; determining the predicted change of the device defect detection model according to the predicted category and the predicted probability includes:

[0014] Obtain the prediction category and prediction probability corresponding to the first moment and the prediction category and prediction probability corresponding to the second moment;

[0015] Determine the minimum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold as the degree of decrease in the predicted probability that the predicted category predicted by the device defect detection model is the actual category;

[0016] Determining the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is a non-actual category according to the maximum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold;

[0017] According to the prediction probability at the first moment, the prediction probability at the second moment, the degree of decrease in the prediction probability and the degree of increase in the prediction probability, the forgetting learning prediction change of the device defect detection model in the forgetting learning direction at the first moment is determined.

[0018] In an embodiment of the present invention, the predicted change includes a forgetting learning predicted change of the device defect detection model in a learning direction; determining the predicted change of the device defect detection model according to the predicted category and the predicted probability includes:

[0019] Obtain the prediction category and prediction probability corresponding to the first moment and the prediction category and prediction probability corresponding to the second moment;

[0020] Determine the maximum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold as the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is the actual category;

[0021] Determining a decrease degree of the predicted probability that the predicted category predicted by the device defect detection model is a non-actual category according to a minimum value of a difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold;

[0022] According to the predicted probability at the first moment, the predicted probability at the second moment, the degree of decrease in the predicted probability and the degree of increase in the predicted probability, the learning predicted change of the device defect detection model in the learning direction at the first moment is determined.

[0023] In one embodiment of the present invention, updating the weight corresponding to the device sample image according to the predicted change includes:

[0024] Construct a difficulty model based on preset prior parameters, forgetting learning prediction changes, and learning prediction changes;

[0025] The weights corresponding to the device sample images are updated according to the difficulty model.

[0026] In one embodiment of the present invention, the device defect detection model to be trained includes a pre-trained feature extractor. After the device sample image is input into the device defect detection model to be trained and the prediction category and prediction probability corresponding to different moments of the device defect detection model output are obtained, the method further includes:

[0027] Obtaining an initial feature map corresponding to the device sample image output by the pre-trained feature extractor;

[0028] Calculating an initial mean value according to the initial feature map and the number of device sample images;

[0029] Get the current feature map of the current batch;

[0030] Calculate the current mean value based on the current feature map;

[0031] Determine a target mean value according to the current mean value and the initial mean value;

[0032] determining a target variance according to the target mean;

[0033] The current feature map of the current batch is processed according to the target mean and the target variance, and the processed current feature map is used. The device defect detection model is used to predict the category and prediction probability corresponding to different moments of the output of the current feature map.

[0034] In one embodiment of the present invention, after acquiring the device sample image, the method further includes:

[0035] The device image is processed according to a preset expansion strategy; wherein the preset expansion strategy at least includes saving the original image, rotation, horizontal mirroring, Gaussian filtering, and vertical mirroring.

[0036] In one embodiment of the present invention, when the device defect detection model satisfies a preset convergence condition, a trained target device defect detection model is obtained, including:

[0037] When the number of iterations of the device defect detection model reaches a preset number of iterations, a trained target device defect detection model is obtained.

[0038] In one embodiment of the present invention, the device defect detection model is a ResNet50 model.

[0039] The embodiment of the present invention further discloses a device defect detection device, the device comprising:

[0040] An acquisition module, used for acquiring device sample images; wherein the device sample images have corresponding actual categories and weights, the device sample images at least include normal device sample images and defective device sample images, the actual categories at least include normal categories and defective categories, and the number of the normal device sample images and the number of the defective device sample images are different;

[0041] An input module, used to input the device sample image into the device defect detection model to be trained, and obtain the prediction category and prediction probability corresponding to the output of the device defect detection model at different times; the prediction probability is the probability that the device defect detection model predicts that the device image is the prediction category;

[0042] A determination module, configured to determine a predicted change of the device defect detection model according to the predicted category and the predicted probability;

[0043] An updating module, used for updating the weight corresponding to the device sample image according to the predicted change;

[0044] A training module, used for training the device defect detection model according to the device sample image with updated weights;

[0045] A generation module is used to obtain a trained target device defect detection model when the device defect detection model meets a preset convergence condition; the target device defect detection model is used to detect the category corresponding to the input device image.

[0046] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0047] The memory is used to store computer programs;

[0048] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0049] The embodiment of the present invention further discloses a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the embodiment of the present invention.

[0050] The embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present invention.

[0051] The embodiments of the present invention include the following advantages:

[0052] In an embodiment of the present invention, when training a device defect detection model to be trained, a device sample image is obtained, wherein the device sample image has a corresponding actual category and weight, the device sample image includes at least a normal device sample image and a defective device sample image, the actual category includes at least a normal category and a defective category, and the number of normal device sample images and defective device sample images is different, the device sample image is input into the device defect detection model to be trained, and the predicted category and predicted probability corresponding to the output of the device defect detection model at different times are obtained, the predicted probability is the probability that the device defect detection model predicts that the device image is the predicted category, the predicted change of the device defect detection model is determined according to the predicted category and the predicted probability, the weight corresponding to the device sample image is updated according to the predicted change, the device defect detection model is trained according to the device sample image with the updated weight, and when the device defect detection model meets the preset convergence condition, the trained target device defect detection model is obtained; the target device defect detection model is used to detect the category corresponding to the input device image. In the embodiment of the present invention, when the number of normal device sample images and defective device sample images is different, for a minority category with high learning difficulty, a relatively high weight is dynamically allocated, so as to improve the recognition performance of the device defect detection model on the minority category, thereby meeting the demand for automatic detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flowchart of a device defect detection method provided in an embodiment of the present invention;

[0054] Figure 2 Schematic diagram of a network structure of a ResNet50 model provided in an embodiment of the present invention;

[0055] Figure 3 is a schematic diagram of a Bottleneck structure provided in an embodiment of the present invention;

[0056] Figure 4 is a structural block diagram of a device defect detection apparatus provided in an embodiment of the present invention;

[0057] Figure 5 It is a schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] In a specific implementation, there is a category imbalance in the sample data set of the device. The model trained with the unbalanced data set often focuses on the large sample category and pays less attention to the small sample category, resulting in low recognition accuracy of the device defect detection model on the small sample category. To address this problem, the embodiment of the present invention proposes an instance-level rebalancing strategy, which dynamically assigns relatively high weights to minority categories with high learning difficulty, thereby improving the recognition performance of the device defect detection model on minority categories. In addition, for small sample data sets, a pre-trained model is usually used for fine-tuning, but there is still a problem of overfitting. The embodiment of the present invention also introduces an adaptive random feature regularization method to effectively alleviate the overfitting of the device defect detection model during the training process.

[0060] Reference Figure 1 , shows a flowchart of a device defect detection method provided in an embodiment of the present invention, which may specifically include the following steps:

[0061] Step 101, obtaining device sample images; wherein the device sample images have corresponding actual categories and weights, the device sample images at least include normal device sample images and defective device sample images, the actual categories at least include normal categories and defective categories, and the number of the normal device sample images and the number of the defective device sample images are different;

[0062] In one embodiment of the present invention, the device at least includes a device on an air conditioning refrigeration system, and the device on the air conditioning refrigeration system can be at least an evaporator and a condenser. Of course, the device in the embodiment of the present invention can also be other devices on the air conditioning refrigeration system, or devices on other devices, and the embodiment of the present invention does not need to be limited to this. For the sake of convenience, the following mainly takes two devices as an example for description, and of course, other devices besides the two devices can be used in practice.

[0063] In an embodiment of the present invention, when performing model training, device sample images (samples) are obtained, wherein the device sample images have corresponding actual categories and weights, and the device sample images may include at least normal device sample images and defective device sample images, wherein the normal device sample images refer to images of normal devices, and the corresponding category is the normal category, while the defective device sample images refer to images of defective devices, and the corresponding category is the defect category. In the actual production process, the number of normal devices is usually much larger than the number of defective devices, resulting in an imbalance in the number of normal device sample images and defective device sample images in the sample data set used for model training, resulting in poor recognition performance of the model on a minority category (i.e., defective device sample images), making it difficult to meet the demand for automatic device detection in the production process.

[0064] Step 102: input the device sample image into the device defect detection model to be trained, and obtain the prediction category and prediction probability corresponding to different moments output by the device defect detection model; the prediction probability is the probability that the device defect detection model predicts that the device image is the prediction category;

[0065] In one embodiment of the present invention, the device defect detection model to be trained may be a ResNet50 model or other models, and the embodiment of the present invention does not need to be limited to this. Specifically, the ResNet50 model is a deep convolutional neural network used for image recognition and classification tasks. ResNet50 is one of the ResNet series models, which solves the gradient vanishing and explosion problems in deep neural networks by introducing residual learning, thereby enabling the network to train deeper structures.

[0066] In an embodiment of the present invention, a device sample image can be input into a device defect detection model to be trained, and the device defect detection model can output the prediction category and prediction probability corresponding to different times t, where the prediction probability is the probability that the device defect detection model predicts that the device image is the prediction category.

[0067] For example, refer to Figure 2 , is a schematic diagram of a network structure of a ResNet50 model provided in an embodiment of the present invention. Figure 2In the ResNet50 model shown in the figure, the two images are passed through a 7×7 convolution layer with a stride of 2 to adjust the number of channels and extract features, and the output size is a feature map of 112×112 with 64 channels; after batch normalization and ReLU function activation, the maximum pooling is used to reduce the size to half of the original size; the number of bottlenecks in the four layers (layer1, layer2, layer3, layer4) is 3, 4, 6, and 3 respectively. The first bottleneck of each layer has a downsample module, and the others are directly connected; finally, the global average pooling technology and dense connection layer are used to output the predicted category. Among them, the Bottleneck structure is a design pattern used to reduce the number of channels of feature maps in deep networks such as ResNet, which can reduce the amount of calculation while maintaining the depth and performance of the network.

[0068] Reference Figure 3 , is a schematic diagram of a Bottleneck structure provided in an embodiment of the present invention, where the dotted line represents the absence of a downsample module. Specifically, Figure 3 In the above, input represents the input feature map; Conv2d represents the two-dimensional convolutional layer; BatchNorm2d+ReLU represents that this layer first performs batch normalization on the output feature map, and then uses the ReLU activation function to activate it; Conv2d+BatchNorm2d represents that this layer first performs convolution on the input feature map, and then batch normalization; Add represents the addition of two feature maps; Output represents the output feature map after being processed by this structure; Bottleneck1 and Bottleneck2 represent two structural forms of Bottleneck. The feature map that passes through the Conv2d+BatchNorm2d layer is called Bottleneck1, and the directly connected one is called Bottleneck2.

[0069] Step 103: Determine a predicted change of the device defect detection model according to the predicted category and the predicted probability.

[0070] Step 104: updating the weight corresponding to the device sample image according to the predicted change;

[0071] Step 105: training the device defect detection model according to the device sample image with updated weights;

[0072] In an embodiment of the present invention, an instance-level balancing strategy is introduced during the training process of the device defect detection model, and a relatively high weight is assigned to minority category samples that are difficult to learn, thereby improving the recognition performance of the device defect detection model on minority categories. Specifically, the instance-level balancing strategy is a technology used to address the problem of class imbalance. The class imbalance problem refers to the situation in which the number of samples of certain categories in a data set is far less than that of other categories, causing the device defect detection model to tend to focus on categories with a large number of samples and ignore categories with a small number of samples during training. The instance-level balancing strategy dynamically adjusts the weight of each sample so that the device defect detection model can pay more attention to those minority category samples that are difficult to learn during the training process, thereby improving the recognition performance of the device defect detection model on minority categories.

[0073] In an embodiment of the present invention, the predicted changes may include the predicted changes of the device defect detection model in the forgetting learning direction, and the predicted changes of the device defect detection model in the learning direction. Specifically, the predicted changes in the forgetting learning direction represent the ability of the device defect detection model to gradually "forget" a certain sample during the training process, while the predicted changes in the learning direction represent the ability of the device defect detection model to gradually "learn" a certain sample during the training process. The embodiment of the present invention can determine the predicted changes of the device defect detection model based on the predicted categories and predicted probabilities output by the device defect detection model at different times, so that the weights corresponding to the device sample images can be updated according to the predicted changes, and then the device defect detection model can be continued to be trained using the device sample images with updated weights, thereby improving the recognition performance of the device defect detection model on a minority category.

[0074] Step 106: When the device defect detection model meets a preset convergence condition, a trained target device defect detection model is obtained; the target device defect detection model is used to detect the category corresponding to the input device image.

[0075] In an embodiment of the present invention, when the device defect detection model satisfies a preset convergence condition, such as the number of iterations reaches a preset number of iterations, or the loss value reaches a preset loss value, it is determined that the training of the device defect detection model is completed, and the device defect detection model at this time can be used as a target device defect detection model. Subsequently, the target device defect detection model can be deployed to a corresponding system or device, so that the target device defect detection model can be used to detect the category corresponding to the device image, and then the device corresponding to the device image can be processed accordingly according to the category.

[0076] In the above-mentioned device defect detection method, when training the device defect detection model to be trained, a device sample image is obtained, wherein the device sample image has a corresponding actual category and weight, the device sample image includes at least a normal device sample image and a defective device sample image, the actual category includes at least a normal category and a defective category, the number of normal device sample images and defective device sample images is different, the device sample image is input into the device defect detection model to be trained, and the prediction category and prediction probability corresponding to the output of the device defect detection model at different times are obtained, the prediction probability is the probability that the device defect detection model predicts that the device image is the prediction category, the prediction change of the device defect detection model is determined according to the prediction category and the prediction probability, the weight corresponding to the device sample image is updated according to the prediction change, the device defect detection model is trained according to the device sample image with the updated weight, and when the device defect detection model meets the preset convergence condition, the trained target device defect detection model is obtained; the target device defect detection model is used to detect the category corresponding to the input device image. In the embodiment of the present invention, when the number of normal device sample images and defective device sample images is different, for a minority category with high learning difficulty, a relatively high weight can be dynamically assigned, so as to improve the recognition performance of the device defect detection model on the minority category, thereby meeting the needs of automatic detection.

[0077] In one embodiment of the present invention, updating the weight corresponding to the device sample image according to the predicted change includes:

[0078] Construct a difficulty model based on preset prior parameters, forgetting learning prediction changes, and learning prediction changes;

[0079] The weights corresponding to the device sample images are updated according to the difficulty model.

[0080] In an embodiment of the present invention, an instance-level rebalancing strategy is introduced on the device defect detection model for device forming quality inspection. By defining a difficulty model, the device sample images of the device defect detection model are resampled, and relatively high weights are assigned to the difficult-to-learn minority category samples, thereby improving the recognition performance of the device defect detection model on minority categories.

[0081] Exemplarily, the definition of the difficulty model may be:

[0082]

[0083] Among them, c is the prior parameter of instance difficulty, which is a constant, du i,t represents the predicted change of the device defect detection model in the forgetting learning direction, dl i,t Represents the predicted change of the device defect detection model in the learning direction.

[0084] After building a difficulty model based on preset prior parameters, forgetting learning prediction changes and learning prediction changes, the weights corresponding to the device sample images can be updated by resampling according to the difficulty model. The formula for updating the weights by resampling can be:

[0085]

[0086] Among them, w i,t represents the weight of category i at time t, D j,t Represents the difficulty model.

[0087] In the above embodiment, a difficulty model is constructed based on preset prior parameters, forgotten learning prediction changes and learned prediction changes, and then the weight corresponding to each device sample image is dynamically updated according to the difficulty model. During the training process, the device defect detection model can pay more attention to those minority category samples that are difficult to learn, thereby improving the recognition performance of the device defect detection model on minority categories.

[0088] In an embodiment of the present invention, the predicted change includes a forgetting learning predicted change of the device defect detection model in a forgetting learning direction; determining the predicted change of the device defect detection model according to the predicted category and the predicted probability includes:

[0089] Obtain the prediction category and prediction probability corresponding to the first moment and the prediction category and prediction probability corresponding to the second moment;

[0090] Determine the minimum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold as the degree of decrease in the predicted probability that the predicted category predicted by the device defect detection model is the actual category;

[0091] Determining the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is a non-actual category according to the maximum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold;

[0092] According to the prediction probability at the first moment, the prediction probability at the second moment, the degree of decrease in the prediction probability and the degree of increase in the prediction probability, the forgetting learning prediction change of the device defect detection model in the forgetting learning direction at the first moment is determined.

[0093] In one embodiment of the present invention, the predicted change may include a forgetting learning predicted change of the device defect detection model in the forgetting learning direction. Exemplarily, the calculation formula of the forgetting learning predicted change of the device defect detection model in the forgetting learning direction is:

[0094]

[0095] Among them,i,t represents the predicted change of the device defect detection model in the forgetting learning direction, represents the predicted probability that the model predicts that sample i is of category j at time t (the first moment), represents the predicted probability of sample i being of category j at time t-1 (second time), min represents the minimum value, max represents the maximum value, y i represents the actual category corresponding to sample i, represents the degree of decrease in the predicted probability that the predicted category predicted by the device defect detection model is the actual category, It indicates the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is other categories.

[0096] In an embodiment of the present invention, the predicted change includes a forgetting learning predicted change of the device defect detection model in a learning direction; determining the predicted change of the device defect detection model according to the predicted category and the predicted probability includes:

[0097] Obtain the prediction category and prediction probability corresponding to the first moment and the prediction category and prediction probability corresponding to the second moment;

[0098] Determine the maximum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold as the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is the actual category;

[0099] Determining a decrease degree of the predicted probability that the predicted category predicted by the device defect detection model is a non-actual category according to a minimum value of a difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold;

[0100] According to the predicted probability at the first moment, the predicted probability at the second moment, the degree of decrease in the predicted probability and the degree of increase in the predicted probability, the learning predicted change of the device defect detection model in the learning direction at the first moment is determined.

[0101] In one embodiment of the present invention, the predicted change may include a learning predicted change of the device defect detection model in a learning direction. Exemplarily, the calculation formula of the learning predicted change of the device defect detection model in the learning direction is:

[0102]

[0103] Among them, dl i,t represents the predicted change of the device defect detection model in the learning direction, represents the predicted probability that the model predicts that sample i is of category j at time t (the first moment), represents the predicted probability of sample i being of category j at time t-1 (second time), min represents the minimum value, max represents the maximum value, y i represents the actual category corresponding to sample i, represents the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is the actual category, It represents the degree of decrease in the predicted probability that the predicted category predicted by the device defect detection model is other categories.

[0104] In the above embodiment, the difficulty model evaluates the learning difficulty of each sample by comprehensively considering the predicted changes in the forgotten learning direction and the predicted changes in the learning direction, and dynamically adjusts the weight of each sample, so that the device defect detection model can pay more attention to those minority category samples that are difficult to learn during the training process, thereby improving the recognition performance of the device defect detection model on minority categories, alleviating the category imbalance problem, and improving the generalization ability of the device defect detection model.

[0105] In one embodiment of the present invention, the device defect detection model to be trained includes a pre-trained feature extractor. After the device sample image is input into the device defect detection model to be trained and the prediction category and prediction probability corresponding to different moments of the device defect detection model output are obtained, the method further includes:

[0106] Obtaining an initial feature map corresponding to the device sample image output by the pre-trained feature extractor;

[0107] Calculating an initial mean value according to the initial feature map and the number of device sample images;

[0108] Get the current feature map of the current batch;

[0109] Calculate the current mean value based on the current feature map;

[0110] Determine a target mean value according to the current mean value and the initial mean value;

[0111] determining a target variance according to the target mean;

[0112] The current feature map of the current batch is processed according to the target mean and the target variance, and the processed current feature map is used. The device defect detection model is used to predict the category and prediction probability corresponding to different moments of the output of the current feature map.

[0113] Among them, the mean refers to the mean of each feature channel in the feature map output by the device defect detection model, and the variance refers to the variance of each feature channel in the feature map output by the device defect detection model.

[0114] In an embodiment of the present invention, an adaptive random feature regularization is introduced to replace batch normalization in a device defect detection model, so as to alleviate the problem of overfitting of the device defect detection model during the training process.

[0115] In one embodiment, the mean u k and variance The calculation formula is:

[0116]

[0117] Among them, u k is the mean of the kth feature channel of the feature map, N K is the number of samples of the kth feature channel, is the feature extracted by the pre-trained feature extractor for sample i before fine-tuning; is the variance of the kth feature channel of the feature map.

[0118] In the embodiment of the present invention, the regularized mean and variance are initialized, the initial mean is calculated, and the mean and variance are updated during the model training process. The update formula for updating the mean and variance can be:

[0119]

[0120] in, represents the target mean, α represents a constant, B k represents the batch size, that is, the number of samples in each batch of data, g φ represents the features extracted by the fine-tuned pre-trained feature extractor for sample i, Indicates the current mean corresponding to a batch.

[0121] In an embodiment of the present invention, first, an initial mean can be calculated based on an initial feature map output by a device defect detection model and the number N of device sample images, and then for a current feature map of a current batch, a current mean can be calculated based on the current feature map, a target mean can be determined based on the current mean and the initial mean, and a target variance can be determined based on the target mean. Finally, the current feature map of the current batch can be processed based on the target mean and the target variance, and the device defect detection model can be used for the processed current feature map to predict categories and prediction probabilities corresponding to different moments output by the current feature map.

[0122] In one embodiment of the present invention, after acquiring the device sample image, the method further includes:

[0123] The device image is processed according to a preset expansion strategy; wherein the preset expansion strategy at least includes saving the original image, rotation, horizontal mirroring, Gaussian filtering, and vertical mirroring.

[0124] In an embodiment of the present invention, after collecting device sample images, the device sample images can be expanded by saving the original image, rotating, horizontally mirroring, Gaussian filtering, and vertically mirroring. The expanded data set is divided into two categories: normal devices and defective devices. In order to reduce the computational complexity of the device defect detection model and facilitate the use of device defect detection models with network structures such as ResNet50, the device sample image size can be cropped to 224×224 pixels and then input into the device defect detection model for training. By processing device images with a preset expansion strategy, a large number of diverse sample images can be generated, which increases the diversity of the model training data set, reduces model overfitting, and improves model quality.

[0125] In one embodiment of the present invention, when the device defect detection model satisfies a preset convergence condition, a trained target device defect detection model is obtained, including:

[0126] When the number of iterations of the device defect detection model reaches a preset number of iterations, a trained target device defect detection model is obtained.

[0127] In an embodiment of the present invention, the number of training rounds of the device defect detection model is set to 50, and the SGD (Stochastic Gradient Descent) with momentum is used for training. SGD randomly extracts a group from the samples, updates once according to the gradient after training, and then extracts another batch and updates again. In the case of a large sample size, it may not be necessary to train all the samples to obtain a model with a loss value within an acceptable range. It should be noted that the model uses a batch of samples for each iteration. When the number of iterations reaches the preset number of iterations, the model training is considered to be completed, and the model at this time can be used as the target device defect detection model for automatic category detection of the device.

[0128] In the above embodiment, the SGD with momentum accelerates the convergence speed of the device defect detection model by introducing a momentum term, and reduces the oscillation during the training process, thereby improving the training efficiency and performance of the device defect detection model.

[0129] In summary, the embodiment of the present invention introduces an instance-level rebalancing strategy on the device defect detection model based on the classic ResNet50 for device forming quality inspection. By defining a difficulty model and performing resampling, a relatively high weight is assigned to the hard-to-learn minority category samples, thereby improving the recognition performance of the device defect detection model on minority categories. In addition, the model parameters are adaptively regularized with random features, and the mean and variance of the parameters are updated during the training process to alleviate the overfitting of the device defect detection model on the minority sample data set. Therefore, the device defect detection model of the embodiment of the present invention is applied to the device forming quality inspection to accurately detect the defects of the device, which has high practical value and application prospects.

[0130] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0131] Reference Figure 4 , shows a structural block diagram of a device defect detection device provided in an embodiment of the present invention, which may specifically include the following modules:

[0132] The acquisition module 401 is used to acquire device sample images; wherein the device sample images have corresponding actual categories and weights, the device sample images at least include normal device sample images and defective device sample images, the actual categories at least include normal categories and defective categories, and the number of normal device sample images and the number of defective device sample images are different;

[0133] An input module 402 is used to input the device sample image into the device defect detection model to be trained, and obtain the prediction category and prediction probability corresponding to the output of the device defect detection model at different times; the prediction probability is the probability that the device defect detection model predicts that the device image is the prediction category;

[0134] A determination module 403, configured to determine a predicted change of the device defect detection model according to the predicted category and the predicted probability;

[0135] An updating module 404, configured to update the weight corresponding to the device sample image according to the predicted change;

[0136] A training module 405 is used to train the device defect detection model according to the device sample image with updated weights;

[0137] The generation module 406 is used to obtain a trained target device defect detection model when the device defect detection model meets a preset convergence condition; the target device defect detection model is used to detect the category corresponding to the input device image.

[0138] In one embodiment of the present invention, the device at least includes an evaporator and a condenser on an air-conditioning refrigeration system.

[0139] In an embodiment of the present invention, the determining module 403 is used to:

[0140] Obtain the prediction category and prediction probability corresponding to the first moment and the prediction category and prediction probability corresponding to the second moment;

[0141] Determine the minimum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold as the degree of decrease in the predicted probability that the predicted category predicted by the device defect detection model is the actual category;

[0142] Determining the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is a non-actual category according to the maximum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold;

[0143] According to the prediction probability at the first moment, the prediction probability at the second moment, the degree of decrease in the prediction probability and the degree of increase in the prediction probability, the forgetting learning prediction change of the device defect detection model in the forgetting learning direction at the first moment is determined.

[0144] In an embodiment of the present invention, the determining module 403 is used to:

[0145] Obtain the prediction category and prediction probability corresponding to the first moment and the prediction category and prediction probability corresponding to the second moment;

[0146] Determine the maximum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold as the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is the actual category;

[0147] Determining a decrease degree of the predicted probability that the predicted category predicted by the device defect detection model is a non-actual category according to a minimum value of a difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold;

[0148] According to the predicted probability at the first moment, the predicted probability at the second moment, the degree of decrease in the predicted probability and the degree of increase in the predicted probability, the learning predicted change of the device defect detection model in the learning direction at the first moment is determined.

[0149] In one embodiment of the present invention, the update module 404 is used to:

[0150] Construct a difficulty model based on preset prior parameters, forgetting learning prediction changes, and learning prediction changes;

[0151] The weights corresponding to the device sample images are updated according to the difficulty model.

[0152] In one embodiment of the present invention, the device further comprises: a processing module, configured to:

[0153] Obtaining an initial feature map corresponding to the device sample image output by the pre-trained feature extractor;

[0154] Calculating an initial mean value according to the initial feature map and the number of device sample images;

[0155] Get the current feature map of the current batch;

[0156] Calculate the current mean value based on the current feature map;

[0157] Determine a target mean value according to the current mean value and the initial mean value;

[0158] determining a target variance according to the target mean;

[0159] The current feature map of the current batch is processed according to the target mean and the target variance, and the processed current feature map is used. The device defect detection model is used to predict the category and prediction probability corresponding to different moments of the output of the current feature map.

[0160] In one embodiment of the present invention, the device further comprises: an expansion module, configured to:

[0161] The device image is processed according to a preset expansion strategy; wherein the preset expansion strategy at least includes saving the original image, rotation, horizontal mirroring, Gaussian filtering, and vertical mirroring.

[0162] In one embodiment of the present invention, the generating module 406 is used to:

[0163] When the number of iterations of the device defect detection model reaches a preset number of iterations, a trained target device defect detection model is obtained.

[0164] In one embodiment of the present invention, the device defect detection model is a ResNet50 model.

[0165] In an embodiment of the present invention, when training a device defect detection model to be trained, a device sample image is obtained, wherein the device sample image has a corresponding actual category and weight, the device sample image includes at least a normal device sample image and a defective device sample image, the actual category includes at least a normal category and a defective category, and the number of normal device sample images and defective device sample images is different, the device sample image is input into the device defect detection model to be trained, and the predicted category and predicted probability corresponding to the output of the device defect detection model at different times are obtained, the predicted probability is the probability that the device defect detection model predicts that the device image is the predicted category, the predicted change of the device defect detection model is determined according to the predicted category and the predicted probability, the weight corresponding to the device sample image is updated according to the predicted change, the device defect detection model is trained according to the device sample image with the updated weight, and when the device defect detection model meets the preset convergence condition, the trained target device defect detection model is obtained; the target device defect detection model is used to detect the category corresponding to the input device image. In the embodiment of the present invention, when the number of normal device sample images and defective device sample images is different, for a minority category with high learning difficulty, a relatively high weight is dynamically allocated, so as to improve the recognition performance of the device defect detection model on the minority category, thereby meeting the demand for automatic detection.

[0166] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] In addition, an embodiment of the present invention further provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned device defect detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0168] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the device defect detection method embodiment described above is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0169] An embodiment of the present invention also provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned device defect detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0170] Figure 5 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0171] The electronic device 500 includes but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510, and a power supply 511. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiments of the present invention, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0172] It should be understood that in the embodiment of the present invention, the radio frequency unit 501 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving downlink data from the base station, it is sent to the processor 510 for processing; in addition, uplink data is sent to the base station. Generally, the radio frequency unit 501 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 501 can also communicate with the network and other devices through a wireless communication system.

[0173] The electronic device provides users with wireless broadband Internet access through the network module 502, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0174] The audio output unit 503 can convert the audio data received by the RF unit 501 or the network module 502 or stored in the memory 509 into an audio signal and output it as sound. Moreover, the audio output unit 503 can also provide audio output related to a specific function performed by the electronic device 500 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 503 includes a speaker, a buzzer, a receiver, etc.

[0175] The input unit 504 is used to receive audio or video signals. The input unit 504 may include a graphics processor (GPU) 5041 and a microphone 5042. The graphics processor 5041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 506. The image frame processed by the graphics processor 5041 can be stored in the memory 509 (or other storage medium) or sent via the radio frequency unit 501 or the network module 502. The microphone 5042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 501 in the case of a telephone call mode.

[0176] The electronic device 500 also includes at least one sensor 505, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 5061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 5061 and / or the backlight when the electronic device 500 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 505 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0177] The display unit 506 is used to display information input by the user or information provided to the user. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0178] The user input unit 507 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 507 includes a touch panel 5071 and other input devices 5072. The touch panel 5071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 5071 or near the touch panel 5071 using any suitable object or accessory such as a finger, stylus, etc.). The touch panel 5071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the contact point coordinates, and then sends it to the processor 510, receives the command sent by the processor 510 and executes it. In addition, the touch panel 5071 can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic waves. In addition to the touch panel 5071, the user input unit 507 may also include other input devices 5072. Specifically, other input devices 5072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail here.

[0179] Furthermore, the touch panel 5071 may be overlaid on the display panel 5061. When the touch panel 5071 detects a touch operation on or near it, it transmits the information to the processor 510 to determine the type of the touch event. Then, the processor 510 provides a corresponding visual output on the display panel 5061 according to the type of the touch event. Figure 5 In the figure, the touch panel 5071 and the display panel 5061 are used as two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 5071 and the display panel 5061 can be integrated to realize the input and output functions of the electronic device, which is not limited here.

[0180] The interface unit 508 is an interface for connecting an external device to the electronic device 500. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 508 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 500 or may be used to transmit data between the electronic device 500 and an external device.

[0181] The memory 509 can be used to store software programs and various data. The memory 509 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 509 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0182] The processor 510 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 509 and calling data stored in the memory 509, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 510 may include one or more processing units; preferably, the processor 510 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 510.

[0183] The electronic device 500 may also include a power supply 511 (such as a battery) for supplying power to each component. Preferably, the power supply 511 may be logically connected to the processor 510 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0184] In addition, the electronic device 500 includes some functional modules not shown, which will not be described in detail here.

[0185] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0186] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0187] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

[0188] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0189] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0190] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0191] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0192] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0193] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0194] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A device defect detection method, characterized in that: The method comprises: Acquire device sample images; wherein the device sample images have corresponding actual categories and weights, the device sample images at least include normal device sample images and defective device sample images, the actual categories at least include normal categories and defective categories, and the number of the normal device sample images and the number of the defective device sample images are different; Input the device sample image into the device defect detection model to be trained, and obtain the prediction category and prediction probability corresponding to the different moments output by the device defect detection model; the prediction probability is the probability that the device defect detection model predicts that the device image is the prediction category; Determining a predicted change of the device defect detection model according to the predicted category and the predicted probability; updating the weight corresponding to the device sample image according to the predicted change; Training the device defect detection model according to the device sample images with updated weights; When the device defect detection model meets the preset convergence condition, a trained target device defect detection model is obtained; the target device defect detection model is used to detect the category corresponding to the input device image.

2. The method according to claim 1, characterized in that The device at least includes an evaporator and a condenser on an air-conditioning refrigeration system.

3. The method according to claim 1, characterized in that The predicted change includes a forgetting learning predicted change of the device defect detection model in a forgetting learning direction; determining the predicted change of the device defect detection model according to the predicted category and the predicted probability includes: Obtain the prediction category and prediction probability corresponding to the first moment and the prediction category and prediction probability corresponding to the second moment; Determine the minimum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold as the degree of decrease in the predicted probability that the predicted category predicted by the device defect detection model is the actual category; Determining the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is a non-actual category according to the maximum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold; According to the prediction probability at the first moment, the prediction probability at the second moment, the degree of decrease in the prediction probability and the degree of increase in the prediction probability, the forgetting learning prediction change of the device defect detection model in the forgetting learning direction at the first moment is determined.

4. The method according to claim 1, characterized in that: The predicted change includes a forgetting learning predicted change of the device defect detection model in a learning direction; determining the predicted change of the device defect detection model according to the predicted category and the predicted probability includes: Obtain the prediction category and prediction probability corresponding to the first moment and the prediction category and prediction probability corresponding to the second moment; Determine the maximum value of the difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold as the degree of increase in the predicted probability that the predicted category predicted by the device defect detection model is the actual category; Determining a decrease degree of the predicted probability that the predicted category predicted by the device defect detection model is a non-actual category according to a minimum value of a difference between the predicted probability at the first moment and the predicted probability at the second moment and a preset threshold; According to the predicted probability at the first moment, the predicted probability at the second moment, the degree of decrease in the predicted probability and the degree of increase in the predicted probability, the learning predicted change of the device defect detection model in the learning direction at the first moment is determined.

5. The method according to claim 3 or 4, characterized in that: Updating the weight corresponding to the device sample image according to the predicted change includes: Construct a difficulty model based on preset prior parameters, forgetting learning prediction changes, and learning prediction changes; The weights corresponding to the device sample images are updated according to the difficulty model.

6. The method according to claim 1, characterized in that The device defect detection model to be trained includes a pre-trained feature extractor. After the device sample image is input into the device defect detection model to be trained and the prediction category and prediction probability corresponding to different moments of the device defect detection model output are obtained, the method further includes: Obtaining an initial feature map corresponding to the device sample image output by the pre-trained feature extractor; Calculating an initial mean value according to the initial feature map and the number of device sample images; Get the current feature map of the current batch; Calculate the current mean value based on the current feature map; Determine a target mean value according to the current mean value and the initial mean value; determining a target variance according to the target mean; The current feature map of the current batch is processed according to the target mean and the target variance, and the processed current feature map is used. The device defect detection model is used to predict the category and prediction probability corresponding to different moments of the output of the current feature map.

7. The method according to claim 1, characterized in that After acquiring the device sample image, the method further includes: The device image is processed according to a preset expansion strategy; wherein the preset expansion strategy at least includes saving the original image, rotation, horizontal mirroring, Gaussian filtering, and vertical mirroring.

8. The method according to claim 1, characterized in that When the device defect detection model meets the preset convergence condition, a trained target device defect detection model is obtained, including: When the number of iterations of the device defect detection model reaches a preset number of iterations, a trained target device defect detection model is obtained.

9. The method according to claim 1, characterized in that: The device defect detection model is a ResNet50 model.

10. A device defect detection apparatus, characterized in that: The device comprises: An acquisition module, used for acquiring device sample images; wherein the device sample images have corresponding actual categories and weights, the device sample images at least include normal device sample images and defective device sample images, the actual categories at least include normal categories and defective categories, and the number of the normal device sample images and the number of the defective device sample images are different; An input module, used to input the device sample image into the device defect detection model to be trained, and obtain the prediction category and prediction probability corresponding to the output of the device defect detection model at different times; the prediction probability is the probability that the device defect detection model predicts that the device image is the prediction category; A determination module, configured to determine a predicted change of the device defect detection model according to the predicted category and the predicted probability; An updating module, used for updating the weight corresponding to the device sample image according to the predicted change; A training module, used for training the device defect detection model according to the device sample image with updated weights; A generation module is used to obtain a trained target device defect detection model when the device defect detection model meets a preset convergence condition; the target device defect detection model is used to detect the category corresponding to the input device image.

11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 9 when executing the program stored in the memory.

12. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 9.

Images