Semi-supervised based welding defect detection method, device, equipment and medium
By applying weak and strong enhancement processing to the welding defect detection method of new energy battery electrode sheets, and combining pseudo-labeling technology with consistency regularization loss to generate model loss, the problem of high-cost labeled data in existing technologies is solved, and efficient welding defect detection is achieved.
Patent Information
- Application Number
- CN202210946387.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-08
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2042-08-08
AI Technical Summary
Existing semi-supervised welding defect detection methods require a large amount of data annotation, resulting in high costs. Furthermore, deep learning models are highly dependent on the labeled data, making it difficult to effectively reduce the cost of welding defect detection for new energy battery electrodes.
By acquiring a small number of labeled welding images and a large number of unlabeled images, weak and strong enhancement processing is performed. Pseudo-labeling technology and consistency regularization loss are combined to generate model loss and update the threshold. The target detection model is then trained, reducing the labeling work and improving the model robustness.
It reduces the cost of detecting welding defects in new energy battery electrode sheets, improves detection accuracy, enhances the robustness of the network, and avoids overfitting and underfitting for different types of defects.
Smart Images
Figure CN115272270B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of new energy automobile battery, and particularly relates to a welding defect detection method and device based on semi-supervision, equipment and medium. BACKGROUND
[0002] The power battery accounts for 30%-40% of the cost of new energy vehicles, and has an important influence on the safety, service life and cruising range of the vehicle. In the production and manufacturing process of the power battery, from the manufacturing of the battery cell to the assembly of the accessories, welding is a very important process. Laser welding has become the preferred method for welding new energy vehicle power batteries due to its flexibility, precision and efficiency. However, in the actual production process, due to factors such as equipment, process, flow and environment, defects such as pores and cracks are inevitable in welding. Therefore, from the perspective of prolonging the service life of the product, or from the perspective of the safety of the user, it is necessary to recheck the welded product to improve the safety performance of the power battery.
[0003] The existing welding defect detection method based on semi-supervision includes traditional image processing and deep learning. Deep learning is superior to traditional visual artificial analysis feature design algorithm due to its strong learning ability, wide coverage and strong adaptability. However, deep learning requires a large amount of data support, and the obtained data cannot be used immediately and needs to be labeled by artificial. The collection, classification and picture labeling of defect data consume a lot of time and money, and when the data is particularly large, it is also necessary to detect whether there is a labeling error to avoid polluting the training result, thereby leading to high cost of welding defect detection based on semi-supervision. There is an urgent need for a method that can reduce the cost of new energy battery electrode welding defect detection. SUMMARY
[0004] The purpose of the embodiment of the present application is to provide a welding defect detection method, device, equipment and medium based on semi-supervision to reduce the detection cost of new energy battery electrode welding defects.
[0005] In order to solve the above technical problems, the embodiment of the present application provides a welding defect detection method based on semi-supervision, comprising:
[0006] Obtain labeled welding images and unlabeled welding images, and perform weak enhancement processing and strong enhancement processing on the unlabeled welding images respectively to obtain weak enhancement images and strong enhancement images, wherein the labeled welding images include welding defect labels;
[0007] Train the detection model through the labeled welding images to obtain an initial detection model, wherein the initial detection model includes a first loss of the labeled welding images and the welding defect labels;
[0008] perform welding defect prediction on the weak-enhanced image and the strong-enhanced image by using the initial detection model to obtain a weak-enhanced image prediction result and a strong-enhanced image prediction result;
[0009] If the weak-enhanced image prediction result is greater than a threshold value, a pseudo label is generated based on the weak-enhanced image prediction result, and cross-entropy loss calculation is performed on the pseudo label and the strong-enhanced image prediction result to obtain a second loss. If the weak-enhanced image prediction result is less than the threshold value, consistency regularization loss calculation is performed based on the weak-enhanced image prediction result and the strong-enhanced image prediction result to obtain a third loss.
[0010] Based on the first loss, the second loss, and the third loss, a model loss is generated, and the threshold value is updated based on the model loss. The step of performing welding defect prediction on the weak-enhanced image and the strong-enhanced image by using the initial detection model is returned until the model loss is less than a first preset value, and a target detection model is obtained.
[0011] If a welding defect detection request is received, a target detection image is obtained, and welding defect prediction is performed on the target detection image by using the target detection model to obtain a target detection result.
[0012] To solve the above technical problems, an embodiment of the present application provides a welding defect detection device based on semi-supervision, comprising:
[0013] A welding image acquisition module is configured to acquire labeled welding images and unlabeled welding images, and perform weak-enhanced processing and strong-enhanced processing on the unlabeled welding images respectively to obtain weak-enhanced images and strong-enhanced images. The labeled welding images include welding defect labels.
[0014] An initial detection model generation module is configured to train a detection model by using the labeled welding images to obtain an initial detection model. The initial detection model includes a first loss of the labeled welding images and the welding defect labels.
[0015] A welding defect prediction module is configured to perform welding defect prediction on the weak-enhanced image and the strong-enhanced image by using the initial detection model to obtain a weak-enhanced image prediction result and a strong-enhanced image prediction result.
[0016] The loss calculation module is configured to, if the weak enhanced image prediction result is greater than a threshold value, generate a pseudo label based on the weak enhanced image prediction result, and perform cross entropy loss calculation on the pseudo label and the strong enhanced image prediction result to obtain a second loss, and if the weak enhanced image prediction result is less than the threshold value, perform consistency regularization loss calculation based on the weak enhanced image prediction result and the strong enhanced image prediction result to obtain a third loss.
[0017] The target detection model generation module is configured to generate a model loss based on the first loss, the second loss, and the third loss, and update the threshold value based on the model loss, and return to perform the step of predicting the welding defects in the weak enhanced image and the strong enhanced image by using the initial detection model until the model loss is less than a first preset value, to obtain a target detection model.
[0018] The target detection result generation module is configured to, if a welding defect detection request is received, obtain a target detection image, and predict welding defects in the target detection image by using the target detection model to obtain a target detection result.
[0019] To solve the above technical problems, one technical solution adopted by the present application is to provide a computer device, which comprises one or more processors and a memory for storing one or more programs, so that the one or more processors implement the semi-supervised welding defect detection method described in any one of the above embodiments.
[0020] To solve the above technical problems, one technical solution adopted by the present application is a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the semi-supervised welding defect detection method described in any one of the above embodiments.
[0021] The embodiments of the present application provide a semi-supervised welding defect detection method, device, equipment and medium. The embodiments of the present application train a detection model by using a small amount of labeled welding images and a large amount of unlabeled welding images, reduce the labeling work of a large amount of welding images, thereby reducing the detection cost of new energy battery pole piece welding defects, and perform weak enhancement and strong enhancement on the unlabeled defect images, enhance the robustness of the network, and change the threshold value for generating a pseudo label in the training process, thereby avoiding overfitting and underfitting of different defect types, and improving the detection accuracy of new energy battery pole piece welding defects. BRIEF DESCRIPTION OF DRAWINGS
[0022] In order to more clearly illustrate the scheme in the application, the drawings needed in the description of the embodiments of the application will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.
[0023] Figure 1 is an implementation flowchart of the semi-supervised based welding defect detection method provided by the embodiments of the application;
[0024] Figure 2 is another implementation flowchart of the sub-process of the semi-supervised based welding defect detection method provided by the embodiments of the application;
[0025] Figure 3 is another implementation flowchart of the sub-process of the semi-supervised based welding defect detection method provided by the embodiments of the application;
[0026] Figure 4 is another implementation flowchart of the sub-process of the semi-supervised based welding defect detection method provided by the embodiments of the application;
[0027] Figure 5 is another implementation flowchart of the sub-process of the semi-supervised based welding defect detection method provided by the embodiments of the application;
[0028] Figure 6 is another implementation flowchart of the sub-process of the semi-supervised based welding defect detection method provided by the embodiments of the application;
[0029] Figure 7 is a schematic diagram of the semi-supervised based welding defect detection device provided by the embodiments of the application;
[0030] Figure 8 is a schematic diagram of the computer device provided by the embodiments of the application. DETAILED DESCRIPTION
[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the description and claims of the application as well as the above drawings, illustrate and set forth rather than limit the application; the terms "include" and "have" and any variations thereof used in the specification and claims of the application and above drawings, are intended to cover the inclusions "not exclusive of other non-recited members". The terms "first", "second" and the like used in the specification and claims of the application and above drawings, are intended to distinguish different objects rather than to describe a particular sequence.
[0032] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. As will be apparent to those of ordinary skill in the art, embodiments described herein can be combinable with other embodiments.
[0033] For those skilled in the art to better understand the scheme of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings.
[0034] The application will be described in detail below in conjunction with the drawings and embodiments.
[0035] It should be noted that the semi-supervised based welding defect detection method provided in the embodiments of the present application is generally executed by a server, and accordingly, the semi-supervised based welding defect detection device is generally configured in the server.
[0036] Please refer to Figure 1 , Figure 1 An embodiment of the semi-supervised based welding defect detection method is shown.
[0037] It should be noted that the method of the present application is not limited to the order of the flow shown, and the method includes the following steps: Figure 1
[0038] S1: Obtain labeled welding images and unlabeled welding images, and perform weak enhancement processing and strong enhancement processing on the unlabeled welding images respectively to obtain weak enhancement images and strong enhancement images, wherein the labeled welding images include welding defect labels.
[0039] Specifically, the welding image refers to an image of a new energy battery pole piece welding position. The labeled welding image refers to labeling the welding image to generate a welding image with a welding defect label. The unlabeled welding image refers to an image that has not been labeled. For the same unlabeled image, weak enhancement processing and strong enhancement processing are performed respectively to obtain weak enhancement images and strong enhancement images.
[0040] It should be noted that the same unlabeled welding image is subjected to weak enhancement processing and strong enhancement processing, so the weak enhancement image and the strong enhancement image are one-to-one corresponding.
[0041] Please refer to Figure 2 , Figure 2 An embodiment of step S2 is shown, described as follows:
[0042] S11: Obtain labeled welding images and unlabeled welding images.
[0043] S12: weakly enhance the unlabeled welding image by adding random noise to obtain an unlabeled welding image.
[0044] S13: horizontally flip the unlabeled welding image to obtain a basic unlabeled welding image.
[0045] S14: perform strong enhancement processing on the basic unlabeled welding image according to a preset operation probability by using multiple image enhancement methods to obtain a strong enhancement image, wherein the image enhancement methods include random Gaussian noise, random disturbance, random shadow, and random illumination.
[0046] Specifically, a small amount of labeled welding images and a large amount of unlabeled welding images are obtained, and then the unlabeled welding images are weakly enhanced by adding random noise to obtain an unlabeled welding image. When performing strong enhancement on the unlabeled welding image, it is horizontally flipped, and then each strong enhancement method such as random Gaussian noise, random disturbance, random shadow, and random illumination is used to perform strong enhancement processing on the basic unlabeled welding image according to a preset operation probability to obtain a strong enhancement image. For example, the operation probability of each strong enhancement method is 50%, and the basic unlabeled welding image is randomly enhanced.
[0047] It should be noted that the preset operation probability is set according to actual conditions, which is not limited here. In a specific embodiment, the operation probability is 50%.
[0048] S2: train the detection model by using the labeled welding images to obtain an initial detection model, wherein the initial detection model includes a first loss of the labeled welding images and welding defect labels.
[0049] Specifically, the labeled welding images are input into the detection model for training, and the detection model is a neural network model, thereby generating an initial detection model.
[0050] Please refer to Figure 3 , Figure 3 An embodiment of step S2 is shown as follows:
[0051] S21: input the labeled welding images into the detection model in batches for welding defect prediction to obtain labeled image prediction results.
[0052] S22: generate a first loss based on the labeled image prediction results and the welding defect labels.
[0053] S23: If the number of training times is less than the preset number or the first loss value is greater than the second preset value, the detection model parameters are adjusted, and the step of inputting the labeled welding image into the detection model in batches for welding defect prediction is performed until the number of training times is equal to the preset number or the first loss value is less than the second preset value, and an initial detection model is obtained.
[0054] Specifically, the labeled welding image is input into the detection model in batches for welding defect prediction to obtain a labeled image prediction result, the labeled image prediction result including a defect category and a position coordinate, and then the defect category and the defect position coordinate are compared with the welding defect label respectively to generate a first loss. If the number of training times is less than the preset number or the first loss value is greater than the second preset value, the detection model parameters are adjusted, and the step S21 is performed by means of back propagation until the number of training times is equal to the preset number or the first loss value is less than the second preset value, and an initial detection model is obtained.
[0055] Further, the calculation formula of the first loss is:
[0056] L SUP = H(P SUP , Label); L SUP represents the first loss, H represents cross-entropy, P SUP represents model output, and Label represents a welding defect label.
[0057] It should be noted that the preset number and the second preset value are set according to actual conditions, and are not limited herein. In a specific embodiment, the preset number is 20, and the second preset value is 0.1.
[0058] S3: Welding defect prediction is performed on the weak enhanced image and the strong enhanced image by means of the initial detection model to obtain a weak enhanced image prediction result and a strong enhanced image prediction result.
[0059] Specifically, the welding defect prediction includes welding defect category prediction and defect position prediction. The weak enhanced image prediction result and the strong enhanced image prediction result include welding defect category prediction results and defect position prediction results. For example, the weak enhanced image prediction result output by the initial detection model is [X1, Y1, X2, Y2, CLASS], in which (X1, Y1) and (X2, Y2) are the left upper and right lower vertices of the defect position coordinate prediction rectangular frame, and are normalized coordinates. CLASS is the predicted welding defect category.
[0060] S4: if the weak enhanced image prediction result is greater than the threshold value, generating a pseudo label based on the weak enhanced image prediction result, and performing cross-entropy loss calculation on the pseudo label and the strong enhanced image prediction result to obtain a second loss, and if the weak enhanced image prediction result is less than the threshold value, performing consistency regularization loss calculation based on the weak enhanced image prediction result and the strong enhanced image prediction result to obtain a third loss.
[0061] Specifically, the weak enhanced image prediction result is compared with the threshold value, if the weak enhanced image prediction result is greater than the threshold value, the weak enhanced image whose weak enhanced image prediction result is greater than the threshold value is taken as the pseudo label, and cross-entropy loss calculation is performed on the pseudo label and the strong enhanced image prediction result to obtain the second loss; and if the weak enhanced image prediction result is less than the threshold value, consistency regularization loss calculation is performed based on the weak enhanced image prediction result and the strong enhanced image prediction result to obtain the third loss.
[0062] Please refer to Figure 4 , Figure 4 An embodiment of step S4 is shown, which is described in detail as follows:
[0063] S41: if the weak enhanced image prediction result is greater than the threshold value, performing mirror processing on the weak enhanced image prediction result to obtain the pseudo label.
[0064] S42: calculating the cross-entropy loss of the pseudo label and the strong enhanced image prediction result to obtain the second loss.
[0065] S43: if the weak enhanced image prediction result is less than the threshold value, calculating the defect category loss and the position loss of the weak enhanced image prediction result and the strong enhanced image prediction result through a preset formula, and performing summation calculation on the defect category loss and the position loss to calculate the consistency regularization loss to obtain the third loss.
[0066] Specifically, the weak enhanced image prediction result includes the welding defect category prediction result and the defect position prediction result, and the weak enhanced image prediction result is processed by normalization, so the weak enhanced image prediction result is compared with the threshold value. The threshold value is preset according to actual needs. Since the strong enhanced image is processed by horizontal flip, if the weak enhanced image prediction result is greater than the threshold value, mirror processing is performed on the weak enhanced image prediction result to obtain the pseudo label, so that the pseudo label and the strong enhanced image position are consistent. Then the cross-entropy loss of the pseudo label and the strong enhanced image prediction result is calculated to obtain the second loss. If the weak enhanced image prediction result is less than the threshold value, the defect category loss and the position loss of the weak enhanced image prediction result and the strong enhanced image prediction result are calculated through a preset formula, and the defect category loss and the position loss are summed to calculate the consistency regularization loss to obtain the third loss.
[0067] In one specific embodiment, the initial detection model outputs a weak enhancement image prediction result of [X1,Y1,X2,Y2,CLASS]. In this weak enhancement image prediction result, (X1,Y1) and (X2,Y2) are the top-left and bottom-right vertices of the defect location coordinate prediction rectangle, and their coordinates are normalized. CLASS is the predicted welding defect category. Because the strong and weak enhancement spaces are horizontally flipped, [1-X2,Y1,1-X1,Y2,CLASS] is used as the pseudo-label of the strong enhancement image prediction result. When the value is less than a threshold, the output of the weak enhancement is assumed to be [X1,Y1,X2,Y2,CLASS]. W ,Y W W W H W CLASS W The strongly enhanced output is [X]. S ,Y S W S H S CLASS S ],(X W ,Y W (W) represents the top left vertex. W H W () represents width and height. After horizontal flipping, only X actually changes. Theoretically, X... S →-X D Therefore, the defect category loss is calculated using the first preset formula.
[0068] The first preset formula is:
[0069] L c =JS(CLASS) W +CLASS S );
[0070] Among them, L C For defect category loss, CLASS W With CLASS S These represent the defect categories in the weakly enhanced image prediction results and the strongly enhanced image prediction results, respectively, with JS representing the JS divergence (Jensen–Shannon Divergence).
[0071] The position loss is calculated using a second preset formula, which is:
[0072] L loc =(X S -(-X W )) 2 +(Y S -Y W ) 2 +(W S -W W) 2 +(H S -H W ) 2 ;
[0073] wherein, L loc is the position loss, (X W , Y W ) and (X S , Y S ) are the top-left vertex coordinates of the weakly enhanced image prediction result and the strongly enhanced image prediction result respectively, (W W , H W ) and (W S , H S ) are the height and width of the weakly enhanced image prediction result and the strongly enhanced image prediction result respectively.
[0074] Further, the second loss is calculated by a third preset formula, wherein the third preset formula is:
[0075] L PSE = H(P strong , Pseudo); L PSE represents the second loss, H represents cross-entropy, P strong represents the model output of the strongly enhanced image, and Pseudo represents the pseudo label.
[0076] In the embodiments of the present application, the pseudo label technology is combined with consistency regularization to reduce the dependence of the model on labeled data, fully utilize unlabeled data, and all pseudo labels are generated online, which is conducive to improving the robustness of the model.
[0077] Please refer to Figure 5 , Figure 5 shows a specific implementation after step S41, which is described as follows:
[0078] S41A: identifying the image dimension identifier whose weakly enhanced image prediction result is greater than the threshold value.
[0079] S41B: based on the image dimension identifier, extracting the strongly enhanced image corresponding to the dimension of the image dimension identifier by a preset extraction method.
[0080] Specifically, since multiple image batches are input into the model during the training process, in the same batch, there may be both images greater than the threshold to generate pseudo labels and images less than the threshold to perform consistency regularization loss, so the two types need to be separated from the batch. Specifically, the image dimension identifier (id) of the weak augmented image greater than the threshold is recorded in the prediction result of the weak augmented image, and the strong augmented image corresponding to the dimension is loaded using the preset extraction method: torch.index_select(strong, 0, ids). The extraction method of the strong augmented image less than the threshold is the same as steps S41A and S41B. To avoid repetition, this will not be repeated here.
[0081] S5: Based on the first loss, the second loss, and the third loss, generate a model loss, and update the threshold based on the model loss, and return to perform the step of predicting the welding defects of the weak augmented image and the strong augmented image by the initial detection model until the model loss is less than the first preset value, and obtain the target detection model.
[0082] Specifically, the threshold is not a fixed value, but is constantly improved as the training progresses, for example, it is improved by 0.05 every 10 training epochs, and the highest is 0.8. At the same time, the thresholds of different defect categories are different, because the detection difficulty of different defects is different, and using the same threshold will cause overfitting of easy-to-detect defect types and underfitting of difficult-to-detect defect types, and the difficult-to-detect threshold is set to half of the ordinary one.
[0083] Please refer to Figure 6 , Figure 6 One specific implementation of step S5 is described in detail as follows:
[0084] S51: Sum the first loss, the second loss, and the third loss to obtain a model loss.
[0085] S52: Determine whether the model loss is less than the first preset value. If not, update the threshold by the preset increase value.
[0086] S53: Separate the weak augmented image and the strong augmented image from the initial detection model.
[0087] S54: Re-input the weak augmented image and the strong augmented image into the initial detection model by the way of back propagation, so that the initial detection model performs welding defect prediction until the model loss is less than the first preset value, and obtains the target detection model.
[0088] Specifically, the first loss, the second loss and the third loss are summed to obtain a model loss, and then it is determined whether the model loss is less than a first preset value. If yes, the model training is stopped to obtain the target detection model. If no, the threshold is updated according to a preset increase value, and then the weak enhanced image and the strong enhanced image are separated from the initial detection model. The weak enhanced image and the strong enhanced image are re-input into the initial detection model in a back propagation manner, so that the initial detection model performs welding defect prediction until the model loss is less than the first preset value, and the target detection model is obtained.
[0089] It should be noted that the first preset value and the preset increase value are set according to actual conditions, which are not limited here.
[0090] S6: If a welding defect detection request is received, a target detection image is obtained, and a welding defect prediction is performed on the target detection image by using the target detection model to obtain a target detection result.
[0091] Specifically, if a welding defect detection request is needed, a welding defect detection request is sent to the server, the server responds to the welding defect detection request, a target detection image is obtained, and a welding defect prediction is performed on the target detection image by using the target detection model to obtain a target detection result.
[0092] In the embodiment, the labeled welding image and the unlabeled welding image are obtained, and the unlabeled welding image is respectively subjected to weak enhancement processing and strong enhancement processing to obtain a weak enhanced image and a strong enhanced image, wherein the labeled welding image comprises a welding defect label; the initial detection model is obtained by training the detection model through the labeled welding image, wherein the initial detection model comprises a first loss of the labeled welding image and the welding defect label; the welding defect prediction of the weak enhanced image and the strong enhanced image is performed through the initial detection model to obtain a weak enhanced image prediction result and a strong enhanced image prediction result; if the weak enhanced image prediction result is greater than a threshold value, the cross-entropy loss calculation is performed based on the weak enhanced image prediction result and the strong enhanced image prediction result to obtain a second loss, and if the weak enhanced image prediction result is less than the threshold value, the consistency regularization loss calculation is performed based on the weak enhanced image prediction result and the strong enhanced image prediction result to obtain a third loss; the model loss is generated based on the first loss, the second loss and the third loss, and the threshold value is updated based on the model loss, and the step of performing the welding defect prediction of the weak enhanced image and the strong enhanced image through the initial detection model is returned to be executed until the model loss is less than a first preset value to obtain a target detection model; if the welding defect detection request is received, the target detection image is obtained, and the welding defect prediction of the target detection image is performed through the target detection model to obtain a target detection result. The welding defect detection model is trained by a small amount of labeled welding images and a large amount of unlabeled welding images, the labeling work of a large amount of welding images is reduced, the detection cost of the new energy battery pole piece welding defect is reduced, the weak enhancement and the strong enhancement of the unlabeled defect image are performed to enhance the robustness of the network, the threshold value of the generated pseudo label is changed in the training process to avoid overfitting and underfitting of different defect types, and the detection accuracy of the new energy battery pole piece welding defect is improved. Furthermore, the pseudo label technology and the consistency regularization are combined to reduce the dependence of the model on the label data, the unlabeled data is fully utilized, all the pseudo labels are generated online, and the robustness of the model is improved.
[0093] Please refer to Figure 7 , as an implementation of the method shown in Figure 1 , the present application provides an embodiment of a welding defect detection device based on semi-supervision, which corresponds to the method embodiment shown in Figure 1 , and the device can be applied to various electronic devices.
[0094] As shown in Figure 7 , the welding defect detection device based on semi-supervision in the embodiment comprises a welding image acquisition module 71, an initial detection model generation module 72, a welding defect prediction module 73, a loss calculation module 74, a target detection model generation module 75 and a target detection result generation module 76, wherein:
[0095] The welding image acquisition module 71 is configured to acquire labeled welding images and unlabeled welding images, and perform weak enhancement processing and strong enhancement processing on the unlabeled welding images respectively to obtain weak enhanced images and strong enhanced images, wherein the labeled welding images comprise welding defect labels;
[0096] The initial detection model generation module 72 is configured to train a detection model by using the labeled welding images to obtain an initial detection model, wherein the initial detection model comprises a first loss of the labeled welding images and the welding defect labels;
[0097] The welding defect prediction module 73 is configured to perform welding defect prediction on the weak enhanced images and the strong enhanced images by using the initial detection model to obtain weak enhanced image prediction results and strong enhanced image prediction results;
[0098] The loss calculation module 74 is configured to, if the weak enhanced image prediction result is greater than a threshold value, generate a pseudo label based on the weak enhanced image prediction result, and perform cross-entropy loss calculation on the pseudo label and the strong enhanced image prediction result to obtain a second loss, and if the weak enhanced image prediction result is less than the threshold value, perform consistency regularization loss calculation on the weak enhanced image prediction result and the strong enhanced image prediction result to obtain a third loss;
[0099] The target detection model generation module 75 is configured to generate a model loss based on the first loss, the second loss and the third loss, update the threshold value based on the model loss, and return to perform the step of performing welding defect prediction on the weak enhanced images and the strong enhanced images by using the initial detection model until the model loss is less than a first preset value to obtain a target detection model;
[0100] The target detection result generation module 76 is configured to, if a welding defect detection request is received, acquire a target detection image, and perform welding defect prediction on the target detection image by using the target detection model to obtain a target detection result.
[0101] Further, the loss calculation module 74 comprises:
[0102] The pseudo label unit is configured to, if the weak enhanced image prediction result is greater than the threshold value, perform mirror processing on the weak enhanced image prediction result to obtain a pseudo label;
[0103] The cross-entropy loss calculation unit is configured to calculate the cross-entropy loss of the pseudo label and the strong enhanced image prediction result to obtain the second loss;
[0104] The consistency regularization loss calculation unit is configured to, if the weak enhanced image prediction result is less than the threshold value, calculate a defect category loss and a position loss of the weak enhanced image prediction result and the strong enhanced image prediction result by using a preset formula, and perform summation calculation on the defect category loss and the position loss to calculate the consistency regularization loss to obtain the third loss.
[0105] wherein the preset formula includes a first preset formula and a second preset formula;
[0106] The first preset formula is:
[0107] L c = JS(CLASS W +CLASS S );
[0108] wherein L C is a defect category loss, CLASS W and CLASS S are defect categories in the weak enhanced image prediction result and the strong enhanced image prediction result respectively, and JS is JS divergence;
[0109] The second preset formula is:
[0110] L loc = (X S - (-X W )) 2 + (Y S - Y W ) 2 + (W S - W W ) 2 + (H S - H W ) 2 ;
[0111] wherein L loc is a position loss, (X W , Y W ) and (X S , Y S ) are top-left coordinates in the weak enhanced image prediction result and the strong enhanced image prediction result respectively, and (W W , H W ) and (W S , H S ) are height and width in the weak enhanced image prediction result and the strong enhanced image prediction result respectively.
[0112] Further, the pseudo label unit further includes:
[0113] an image dimension identifier identifying unit, configured to identify an image dimension identifier of which the weak enhanced image prediction result is greater than a threshold value;
[0114] a strong enhanced image extraction unit, configured to extract a strong enhanced image corresponding to the dimension of the image dimension identifier based on the image dimension identifier and through a preset extraction manner.
[0115] Further, the target detection model generation module 75 includes:
[0116] a model loss calculation unit configured to sum the first loss, the second loss, and the third loss to obtain a model loss;
[0117] a threshold updating unit configured to determine whether the model loss is less than a first preset value, and if not, update the threshold according to a preset increase value;
[0118] an image abstraction unit configured to abstract a weak enhanced image and a strong enhanced image from the initial detection model;
[0119] an iterative calculation unit configured to re-input the weak enhanced image and the strong enhanced image into the initial detection model in a manner of back propagation, so that the initial detection model performs welding defect prediction until the model loss is less than the first preset value, to obtain a target detection model.
[0120] Further, the welding image acquisition module 71 comprises:
[0121] an image acquisition unit configured to acquire a labeled welding image and an unlabeled welding image;
[0122] a weak enhancement processing unit configured to perform weak enhancement processing on the unlabeled welding image by increasing random noise to obtain a weak enhanced unlabeled welding image;
[0123] a horizontal flip processing unit configured to perform horizontal flip processing on the unlabeled welding image to obtain a basic unlabeled welding image;
[0124] a strong enhancement processing unit configured to perform strong enhancement processing on the basic unlabeled welding image by using multiple image enhancement methods according to a preset operation probability to obtain a strong enhanced image, wherein the image enhancement methods include random Gaussian noise, random disturbance, random shadow, and random illumination.
[0125] Further, the initial detection model generation module 72 comprises:
[0126] a labeled image prediction result generation unit configured to input the labeled welding image into the detection model in batches to perform welding defect prediction to obtain a labeled image prediction result;
[0127] a first loss generation unit configured to generate a first loss based on the labeled image prediction result and a welding defect label;
[0128] a detection model parameter adjustment unit configured to adjust the detection model parameters if the number of training times is less than a preset number of times or the first loss value is greater than a second preset value, and return to perform the step of inputting the labeled welding image into the detection model in batches to perform welding defect prediction until the number of training times is equal to the preset number of times or the first loss value is less than the second preset value, to obtain an initial detection model.
[0129] To solve the above technical problems, the embodiments of the present application further provide a computer device. For details, please refer to Figure 8 , Figure 8 The basic structure block diagram of the computer device of the present embodiment is shown in FIG. 1.
[0130] The computer device 8 comprises a memory 81, a processor 82, and a network interface 83 which are connected to each other through a system bus. It should be noted that only the computer device 8 with three components, i.e., the memory 81, the processor 82, and the network interface 83 is shown in the figure, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be alternatively implemented. Among them, those skilled in the art can understand that the computer device herein is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0131] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The computer device can interact with the user through a keyboard, a mouse, a remote controller, a touchpad, a voice control device, and the like.
[0132] The memory 81 includes at least one type of readable storage medium, including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 81 can be an internal storage unit of the computer device 8, such as a hard disk or a memory of the computer device 8. In other embodiments, the memory 81 can also be an external storage device of the computer device 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 8. Of course, the memory 81 can also include both the internal storage unit and the external storage device of the computer device 8. In the present embodiment, the memory 81 is generally used to store an operating system and various application software installed on the computer device 8, such as program codes of the semi-supervised welding defect detection method, etc. In addition, the memory 81 can also be used to temporarily store various data that have been output or will be output.
[0133] The processor 82 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 82 is generally used to control the overall operation of the computer device 8. In the present embodiment, the processor 82 is used to run program codes or process data stored in the memory 81, such as running program codes of the semi-supervised welding defect detection method described above, to implement various embodiments of the semi-supervised welding defect detection method.
[0134] The network interface 83 can include a wireless network interface or a wired network interface, and is generally used to establish a communication connection between the computer device 8 and other electronic devices.
[0135] The present application also provides another embodiment, i.e., to provide a computer readable storage medium storing a computer program, which can be executed by at least one processor to make the at least one processor execute the steps of a semi-supervised welding defect detection method as described above.
[0136] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the method of each embodiment of the present application.
[0137] Obviously, the above-described embodiments are only some of the embodiments of the present application, not all the embodiments, and the drawings show the preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms, and conversely, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing specific embodiments, or make equivalent replacements to some of the technical features. Any equivalent structure made by using the content of the specification and drawings, directly or indirectly applied to other related technical fields, is also within the scope of the patent protection of the present application.
Claims
1. A semi-supervised based welding defect detection method, characterized in that, The method comprises the following steps: obtain labeled welding images and unlabeled welding images, and perform weak enhancement processing and strong enhancement processing on the unlabeled welding images respectively to obtain weak enhancement images and strong enhancement images, wherein the labeled welding images include welding defect labels; train a detection model through the labeled welding images to obtain an initial detection model, wherein the initial detection model includes a first loss of the labeled welding images and the welding defect labels; perform welding defect prediction on the weak enhancement images and the strong enhancement images through the initial detection model to obtain weak enhancement image prediction results and strong enhancement image prediction results; if the weak enhancement image prediction result is greater than a threshold value, generate a pseudo label based on the weak enhancement image prediction result, and perform cross-entropy loss calculation on the pseudo label and the strong enhancement image prediction result to obtain a second loss, and if the weak enhancement image prediction result is less than the threshold value, perform consistency regularization loss calculation based on the weak enhancement image prediction result and the strong enhancement image prediction result to obtain a third loss; generate a model loss based on the first loss, the second loss, and the third loss, update the threshold value based on the model loss, and return to perform the step of performing welding defect prediction on the weak enhancement images and the strong enhancement images through the initial detection model until the model loss is less than a first preset value to obtain a target detection model; if a welding defect detection request is received, obtain a target detection image, and perform welding defect prediction on the target detection image through the target detection model to obtain a target detection result; wherein if the weak enhancement image prediction result is greater than a threshold value, a pseudo label is generated based on the weak enhancement image prediction result, and cross-entropy loss calculation is performed on the pseudo label and the strong enhancement image prediction result to obtain a second loss, and if the weak enhancement image prediction result is less than the threshold value, consistency regularization loss calculation is performed based on the weak enhancement image prediction result and the strong enhancement image prediction result to obtain a third loss, comprising: if the weak enhancement image prediction result is greater than the threshold value, mirror processing is performed on the weak enhancement image prediction result to obtain the pseudo label; calculate the cross-entropy loss of the pseudo label and the strong enhancement image prediction result to obtain the second loss; if the weak enhancement image prediction result is less than the threshold value, calculate the defect category loss and the position loss of the weak enhancement image prediction result and the strong enhancement image prediction result through a preset formula, and perform summation calculation on the defect category loss and the position loss to calculate the consistency regularization loss to obtain the third loss; the preset formula includes a first preset formula and a second preset formula; the first preset formula is: ； wherein, is the defect class loss for the defect class, and are the defect classes in the weak and strong augmented image predictions, respectively, and JS is the JS divergence. the second preset formula is: ； in, For the position loss, and These are the coordinates of the top-left vertex in the weakly enhanced image prediction result and the strongly enhanced image prediction result, respectively. and These are the height and width in the weakly enhanced image prediction result and the strongly enhanced image prediction result, respectively; The first loss, the second loss and the third loss are summed to obtain the model loss. If the model loss is not less than the first preset value, the threshold is updated by a preset increase value. The weak enhanced image and the strong enhanced image are separated from the initial detection model. The weak enhanced image and the strong enhanced image are re-input into the initial detection model through back propagation, so that the initial detection model performs welding defect prediction until the model loss is less than the first preset value, and the target detection model is obtained. If the weak enhanced image prediction result is greater than the threshold, the weak enhanced image prediction result is mirror processed to obtain a pseudo label.
2. The semi-supervised based welding defect detection method of claim 1, wherein, An image dimension identifier is identified. Based on the image dimension identifier, a strong enhanced image corresponding to the dimension of the image dimension identifier is extracted through a preset extraction method. The labeled welding image and the unlabeled welding image are obtained, and the unlabeled welding image is weakly enhanced and strongly enhanced to obtain a weak enhanced image and a strong enhanced image.
3. The semi-supervised based welding defect detection method of claim 1, wherein, The labeled welding image and the unlabeled welding image are obtained. The unlabeled welding image is weakly enhanced by adding random noise to obtain the unlabeled welding image. The unlabeled welding image is horizontally flipped to obtain a basic unlabeled welding image. A plurality of image enhancement methods are used to strongly enhance the basic unlabeled welding image according to a preset operation probability, to obtain the strong enhanced image, wherein the image enhancement methods include random Gaussian noise, random disturbance, random shadow and random illumination. The labeled welding image is input into the detection model in batches to perform welding defect prediction, to obtain a labeled image prediction result.
4. The semi-supervised based welding defect detection method according to any one of claims 1 to 3, characterized in that, Based on the labeled image prediction result and the welding defect label, the first loss is generated. If the training frequency is less than a preset frequency or the first loss value is greater than a second preset value, the detection model parameters are adjusted, and the step of inputting the labeled welding image into the detection model in batches to perform welding defect prediction is returned until the training frequency is equal to the preset frequency or the first loss value is less than the second preset value, to obtain the initial detection model. 5. A semi-supervised based welding defect detection apparatus, characterized by, The welding image acquisition module is configured to acquire labeled welding images and unlabeled welding images, and perform weak enhancement processing and strong enhancement processing on the unlabeled welding images respectively to obtain weak enhanced images and strong enhanced images, wherein the labeled welding images include welding defect labels; The initial detection model generation module is configured to train a detection model by using the labeled welding images to obtain an initial detection model, wherein the initial detection model includes a first loss of the labeled welding images and the welding defect labels; The welding defect prediction module is configured to perform welding defect prediction on the weak enhanced images and the strong enhanced images by using the initial detection model to obtain weak enhanced image prediction results and strong enhanced image prediction results; The loss calculation module is configured to, if the weak enhanced image prediction result is greater than a threshold value, generate a pseudo label based on the weak enhanced image prediction result, and perform cross-entropy loss calculation on the pseudo label and the strong enhanced image prediction result to obtain a second loss, and if the weak enhanced image prediction result is less than the threshold value, perform consistency regularization loss calculation based on the weak enhanced image prediction result and the strong enhanced image prediction result to obtain a third loss; The target detection model generation module is configured to generate a model loss based on the first loss, the second loss, and the third loss, and update the threshold value based on the model loss, and return to perform the welding defect prediction on the weak enhanced images and the strong enhanced images by using the initial detection model until the model loss is less than a first preset value to obtain a target detection model; The target detection result generation module is configured to, if a welding defect detection request is received, acquire a target detection image, and perform welding defect prediction on the target detection image by using the target detection model to obtain a target detection result; The loss calculation module includes: The pseudo label unit is configured to, if the weak enhanced image prediction result is greater than the threshold value, perform mirror processing on the weak enhanced image prediction result to obtain the pseudo label; The cross-entropy loss calculation unit is configured to calculate the cross-entropy loss of the pseudo label and the strong enhanced image prediction result to obtain the second loss; The consistency regularization loss calculation unit is configured to, if the weak enhanced image prediction result is less than the threshold value, calculate a defect category loss and a position loss of the weak enhanced image prediction result and the strong enhanced image prediction result by using a preset formula, and perform summation calculation on the defect category loss and the position loss to calculate the consistency regularization loss to obtain the third loss; The preset formula includes a first preset formula and a second preset formula; The first preset formula is: ； wherein, is the defect class loss for the defect class, and are the defect classes in the weak and strong augmented image predictions, respectively, and JS is the JS divergence. The second preset formula is: ； wherein, is the position loss, and are the top-left coordinates of the weakly and strongly enhanced image prediction results, respectively, and are the height and width of the weakly and strongly enhanced image prediction results, respectively. The target detection model generation module includes: The model loss calculation unit is configured to perform summation processing on the first loss, the second loss, and the third loss to obtain the model loss; The threshold value updating unit is configured to determine whether the model loss is less than the first preset value, and if not, update the threshold value by a preset increase value. an image abstraction unit, configured to abstract the weakly augmented image and the strongly augmented image from the initial detection model; an iterative calculation unit, configured to re-input the weakly augmented image and the strongly augmented image into the initial detection model in a manner of back propagation, so that the initial detection model performs welding defect prediction until the model loss is less than a first preset value, to obtain the target detection model.
6. A computer device, comprising: A computer readable storage medium, having a computer program stored thereon, wherein the computer program is executed by a processor to implement the semi-supervised based welding defect detection method according to any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that, A computer readable storage medium, having a computer program stored thereon, wherein the computer program is executed by a processor to implement the semi-supervised based welding defect detection method according to any one of claims 1 to 4.