Sealing nail welding defect detection method and apparatus, and device and storage medium

Abstract

Disclosed in the present application are a sealing nail welding defect detection method and apparatus, and a device and a storage medium. The method comprises: acquiring a target image to be processed, wherein the target image comprises a sealing nail welding image captured for a sealing nail that is welded using a target welding type; on the basis of target visual domain prompt information, processing the target image to obtain a target prompt image, wherein the target visual domain prompt information comprises visual domain prompt information determined and obtained on the basis of a target domain image set, and the target domain image set comprises a plurality of sealing nail welding images corresponding to the target welding type; and using a defect detection model to perform sealing nail welding defect detection on the target prompt image to obtain a target detection result, wherein the defect detection model comprises a model obtained by means of pre-training using domain image sets other than the target domain image set. The solution provided in the embodiments of the present application can improve the generalization of models, thus improving the detection accuracy of the models.

Description

Sealing nail welding defect detection method, device, equipment and storage medium CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Chinese patent application No. 202410116762.1, filed on January 29, 2024, entitled “Sealing nail welding defect detection method, device, equipment and storage medium,” and the entire contents of that application are incorporated herein by reference. Technical Field

[0002] The present application relates to the technical field of industrial visual defect detection, and in particular to a sealing nail welding defect detection method, device, equipment and storage medium. Background Art

[0003] In the field of new energy technology, sealing pins are widely used in various lithium battery industrial manufacturing scenarios to ensure the sealing and safety of lithium battery products. However, welding defects are inevitable during the sealing pin welding process. Improving the accuracy of sealing pin welding defect detection results is a key prerequisite for improving the sealing and safety of battery products.

[0004] At present, the defect detection model used to detect sealing nail welding defects in the relevant technology is trained using the original sealing nail welding image sample set. However, the model has poor generalization when faced with sealing nail welding images of new welding types that have not been seen during model training, resulting in low model detection accuracy. Summary of the Invention

[0005] The present application provides a sealing pin welding defect detection method, device, equipment and storage medium, which can improve the generalization of the model and thus improve the model detection accuracy.

[0006] In the first aspect, the present application provides a sealing nail welding defect detection method, the method comprising: obtaining a target image to be processed, wherein the target image includes a sealing nail welding image collected for a sealing nail welded using a target welding type; processing the target image according to target visual domain prompt information to obtain a target prompt image, wherein the target visual domain prompt information includes visual domain prompt information determined based on a target domain image set, and the target domain image set includes multiple sealing nail welding images corresponding to the target welding type; performing sealing nail welding defect detection on the target prompt image using a defect detection model to obtain a target detection result, wherein the defect detection model includes a model pre-trained using image sets in other domains other than the target domain image set.

[0007] In this way, by using the sealing nail welding images collected for sealing nails welded using the target welding type in the target domain image set, the target visual domain prompt information corresponding to the target domain image set is determined, and then when the defect detection model pre-trained by the other domain image set is used to perform defect detection on the target image corresponding to the target welding type, the target image can be first processed according to the target visual domain prompt information, and then the defect detection model can be used to perform welding defect detection on the processed target prompt image. In this way, since the image-level visual domain prompt for the target welding type is used when the pre-trained defect detection model is used to perform defect detection on the sealing nail welding image of the new target welding type, the generalization of the model in the target welding type sealing nail welding image defect detection can be improved, and the model detection accuracy can be improved.

[0008] In some embodiments, before processing the target image according to the target visual domain prompt information to obtain the target prompt image, the method also includes: acquiring a first image from the target domain image set; processing the first image according to the first visual domain prompt information to obtain a first prompt image; using the defect detection model to perform sealing nail welding defect detection on the first prompt image to obtain a first prediction result; smoothing the first visual domain prompt information to obtain second visual domain prompt information; processing the second image according to the second visual domain prompt information to obtain a second prompt image, and the second image is determined based on the first image; performing sealing nail welding defect detection on the second prompt image using the defect detection model to obtain a second prediction result; based on the difference between the first prediction result and the second prediction result, updating the first visual domain prompt information to obtain target visual domain prompt information.

[0009] In this way, by adopting a teacher-student framework, visual domain prompt information is gradually updated in an unsupervised training manner, thereby achieving continuous adaptation to new target domain image sets. In addition, since a large number of sealing nail welding images obtained in real industrial scenarios are unlabeled, and the image labeling process is time-consuming, labor-intensive, and costly, the unsupervised training method adopted in the embodiments of the present application can also reduce the data labeling process, saving time, human and material resources, and reducing costs.

[0010] In some embodiments, the smoothing of the first visual domain prompt information to obtain the second visual domain prompt information includes: obtaining the target visual domain prompt information obtained by historical updates to obtain historical visual domain prompt information; based on the historical visual domain prompt information, smoothing the first visual domain prompt information according to an exponential moving average algorithm to obtain the second visual domain prompt information.

[0011] In this way, the first visual domain prompt information is smoothed by the above-mentioned exponential moving average algorithm to obtain the second visual domain prompt information, which can make the optimization effect of the second visual domain prompt information more obvious, and then a more accurate second detection result can be obtained based on the second visual domain prompt information, thereby improving the training and updating efficiency of the visual domain prompt information.

[0012] In some embodiments, before processing the second image according to the second visual domain prompt information, the method further includes: performing enhancement processing on the first image to obtain the second image.

[0013] In this way, by using the second visual domain cue information to process the second image that is similar to the first image, the robustness of the subsequently updated visual domain cue information can be improved.

[0014] In some embodiments, the target visual domain prompt information includes target domain-specific prompt information and target domain-irrelevant prompt information; updating the first visual domain prompt information based on the difference between the first prediction result and the second prediction result to obtain the target visual domain prompt information includes: determining a domain-specific prompt loss value and a domain-irrelevant prompt loss value based on the first prediction result and the second prediction result; updating the domain-specific prompt information in the first visual domain prompt information according to the domain-specific prompt loss value to obtain the target domain-specific prompt information; updating the domain-irrelevant prompt information in the first visual domain prompt information according to the domain-irrelevant prompt loss value to obtain the target domain-irrelevant prompt information.

[0015] In this way, by using both domain-specific and domain-independent cues, we can effectively adapt to the ever-changing target domain image collection. The domain-specific cues help capture domain-specific knowledge, while the domain-independent cues help preserve domain-shared knowledge, reducing the risks of overfitting, catastrophic forgetting, and error accumulation that may arise during the adaptation process.

[0016] In some embodiments, a domain-independent hint loss value is determined based on the first prediction result and the second prediction result, including: determining a first loss value based on the first prediction result and the second prediction result; obtaining a domain-specific hint loss value and a domain-independent hint loss value corresponding to a historical domain image set to obtain a historical visual domain hint loss value; determining a steady-state regularization loss value corresponding to the target domain image set based on the historical visual domain hint loss value; and determining a domain-independent hint loss value based on the first loss value and the steady-state regularization loss value.

[0017] In this way, by using an additional steady-state regularization loss value in the process of updating domain-independent hint information, the steady-state regularization mechanism can be utilized to limit the sensitivity of parameters in the domain-independent hint information to domain changes, ensuring that the domain-independent hint information does not over-adapt to any single domain, but instead captures more general and transferable knowledge, thereby promoting the domain-independent hint information to learn more general knowledge.

[0018] In some embodiments, determining the steady-state regularization loss value corresponding to the target domain image set based on the historical visual domain prompt loss value includes: determining the parameter importance of each parameter in the visual domain prompt information corresponding to the historical domain image set based on the historical visual domain prompt loss value; determining the steady-state factor corresponding to each parameter in the target visual domain prompt information based on the parameter importance of each parameter in the visual domain prompt information corresponding to the historical domain image set and the difference value of each parameter at different times; and determining the steady-state regularization loss value of the target visual domain prompt information based on the steady-state factor.

[0019] In this way, the steady-state regularization loss value can be calculated through the above process to penalize the update of sensitive parameters, prevent domain-independent prompt information from over-adapting to a specific field, and thus promote its learning of more general knowledge.

[0020] In some embodiments, after using the defect detection model to perform sealing nail welding defect detection on the target prompt image and obtaining the target detection result, the method also includes: obtaining the target prediction confidence output by the defect detection model for the target detection result; when the difference between the target prediction confidence and the historical prediction confidence is greater than a preset threshold, updating the steady-state regularization loss value.

[0021] In this way, through the above-mentioned domain shift detection process, the model can automatically detect new domains and then continuously adapt to the new domain image set, further improving the model's automatic generalization ability.

[0022] In the second aspect, the present application provides a sealing nail welding defect detection device, which includes: an image acquisition module for acquiring a target image to be processed, wherein the target image includes a sealing nail welding image collected for a sealing nail welded using a target welding type; an image processing module for processing the target image according to target visual domain prompt information to obtain a target prompt image, wherein the target visual domain prompt information includes visual domain prompt information determined based on a target domain image set, and the target domain image set includes multiple sealing nail welding images corresponding to the target welding type; a defect detection module for performing sealing nail welding defect detection on the target prompt image using a defect detection model to obtain a target detection result, wherein the defect detection model includes a model pre-trained using image sets in other domains other than the target domain image set.

[0023] In a third aspect, the present application provides an electronic device comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the steps of the sealing pin welding defect detection method as described in any embodiment of the first aspect are implemented.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the steps of the sealing pin welding defect detection method as described in any one of the embodiments of the first aspect are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] FIG1 is a schematic flow chart of a sealing pin welding defect detection method provided by one embodiment of the present application;

[0027] FIG2 is a schematic diagram of a sealing pin welding image provided by the present application;

[0028] FIG3 is a schematic diagram of a model detection process provided by the present application;

[0029] FIG4 is a flow chart of a method for detecting sealing pin welding defects according to an embodiment of the present application;

[0030] FIG5 is a flow chart of a method for detecting sealing pin welding defects according to an embodiment of the present application;

[0031] FIG6 is a schematic diagram of a training update process provided by the present application;

[0032] FIG7 is a flow chart of a method for detecting sealing pin welding defects according to an embodiment of the present application;

[0033] FIG8 is a flow chart of a method for detecting sealing pin welding defects according to an embodiment of the present application;

[0034] FIG9 is a flow chart of a method for detecting sealing pin welding defects according to an embodiment of the present application;

[0035] FIG10 is a schematic structural diagram of a sealing pin welding defect detection device provided by one embodiment of the present application;

[0036] FIG11 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0039] Currently, the method used in related technologies for detecting sealing pin weld defects typically involves training a model on a large number of labeled sealing pin weld image samples. After obtaining the trained model, it is used to detect defects in sealing pin weld images. However, due to the wide variety of sealing pin weld types, the model's generalization is poor when detecting defects in sealing pin weld images with new weld types that were not seen during training, resulting in low detection accuracy.

[0040] In order to solve the above technical problems, the embodiments of the present application provide a sealing nail welding defect detection method, device, equipment and storage medium. Among them, the sealing nail welding defect detection method provided by the embodiments of the present application determines the target visual domain prompt information corresponding to the target domain image set by using the sealing nail welding images collected for sealing nails welded with the target welding type in the target domain image set, and then when using the defect detection model pre-trained by other domain image sets to perform defect detection on the target image corresponding to the target welding type, the target image can be first processed according to the target visual domain prompt information, and then the defect detection model can be used to perform welding defect detection on the processed target prompt image. In this way, since the image-level visual domain prompt for the target welding type is used when the pre-trained defect detection model is used to perform defect detection on the sealing nail welding image of the new target welding type, the generalization of the model in the target welding type sealing nail welding image defect detection can be improved, and the model detection accuracy can be improved.

[0041] The following is a detailed introduction to the sealing pin welding defect detection method provided in an embodiment of the present application. This sealing pin welding defect detection method can be applied to the application scenario of detecting sealing pin welding defects. This sealing pin welding defect detection method can be executed by an electronic device with computing capabilities. The electronic device can include terminal devices such as mobile phones and computers, and can also include devices such as servers.

[0042] FIG1 is a flow chart of a method for detecting welding defects of sealing pins provided by an embodiment of the present application. As shown in FIG1 , the method for detecting welding defects of sealing pins may specifically include the following steps:

[0043] S110, acquiring a target image to be processed, wherein the target image includes a sealing nail welding image collected for a sealing nail welded using a target welding type;

[0044] S120, processing the target image according to the target visual domain prompt information to obtain a target prompt image, wherein the target visual domain prompt information includes visual domain prompt information determined based on a target domain image set, and the target domain image set includes a plurality of sealing nail welding images corresponding to the target welding type;

[0045] S130. Perform sealing nail welding defect detection on the target prompt image using a defect detection model to obtain a target detection result, wherein the defect detection model includes a model pre-trained using image sets in other fields other than the target field image set.

[0046] Thus, by using the sealing nail welding images collected for sealing nails welded using the target welding type in the target domain image set, the target visual domain prompt information corresponding to the target domain image set is determined, and then when the defect detection model pre-trained by the other domain image set is used to perform defect detection on the target image corresponding to the target welding type, the target image can be first processed according to the target visual domain prompt information, and then the defect detection model can be used to perform welding defect detection on the processed target prompt image. In this way, since the image-level visual domain prompt for the target welding type is used when the pre-trained defect detection model is used to perform defect detection on the sealing nail welding image of the new target welding type, the generalization of the model in the target welding type sealing nail welding image defect detection can be improved, and the model detection accuracy can be improved.

[0047] The specific implementation methods of the above steps are introduced below.

[0048] In some embodiments, in S110, the target image may be a sealing pin welding image captured for a sealing pin welded using a target welding type, such as the sealing pin welding image shown in FIG2 . The target welding type may be any welding type other than the welding types already included in the training samples used in pre-training of the defect detection model.

[0049] In some embodiments, in S120, the target domain image set may include multiple sealing pin welding images, such as the sealing pin welding images shown in Figure 2. The sealing pin welding images in the target domain image set may be sealing pin welding images collected for sealing pins welded using the target welding type.

[0050] For example, when it is necessary to use a trained defect detection model to perform defect detection on sealing pin welding images corresponding to a new target welding type, a portion of images can be obtained from the target domain image set corresponding to the target welding type to learn the image-level visual cues corresponding to the target domain image set, so that the model can better adapt to the target domain image set based on the cues.

[0051] In addition, the target domain image set may be dynamically changed, and the target domain image set may be updated according to the latest acquired sealing nail welding image.

[0052] In some embodiments, the target visual domain hint information can be used to highlight features unique to the target domain image set, as well as features shared across different domain image sets. This visual domain hint information can help the defect detection model continuously adapt to domain image sets corresponding to different welding types, improving the model's generalization.

[0053] In some examples of the present application, the visual domain prompt information may specifically include domain-specific prompt information and domain-independent prompt information. Among them, the domain-specific prompt information, also known as DSP (Domain-Specific Prompt) information, is intended to extract knowledge specific to a specific domain from a set of images in that domain, and guide the model to focus on specific features and patterns in that domain during the adaptation process. The domain-independent prompt information, also known as DAP (Domain-Agnostic Prompt) information, is intended to maintain domain-shared knowledge in the process of continuously adapting to image sets in different domains, and to help the model retain consistent features and patterns between different domains.

[0054] For example, the target visual domain cue information can be a learnable parameter matrix that changes as the features of the images contained in the target domain image set change, thereby achieving continuous adaptation to the target domain image set. For example, a portion of images from the target domain image set can be used as training samples to perform unsupervised training on the model, thereby gradually updating the parameters contained in the target visual domain cue information to achieve adaptation to the sealing nail welding images corresponding to the target weld type.

[0055] In addition, in some embodiments, after obtaining the target visual domain prompt information corresponding to the target domain image set, the target image can be processed according to the target visual domain prompt information. The processing method can include adding the parameter matrix in the target visual domain prompt information and the feature matrix corresponding to the target image point by point, thereby constructing a target prompt image.

[0056] Exemplarily, as shown in Figure 3, when the target visual domain prompt information 31 includes target domain specific prompt information 311 and target domain irrelevant prompt information 312, the parameter matrix corresponding to the target domain specific prompt information 311 and the parameter matrix corresponding to the target domain irrelevant prompt information 312 can be added point by point to the feature matrix corresponding to the target image, so as to apply the target domain specific prompt information 311 and the target domain irrelevant prompt information 312 to the target image 32, and construct a target prompt image 33 superimposed with the target domain specific prompt information 311 and the target domain irrelevant prompt information 312.

[0057] In some embodiments, in S130, the defect detection model may be a model pre-trained using an image set from a domain other than the target domain image set. The image set from the other domain may include sealing nail weld images corresponding to any one or more weld types other than the target weld type. In other words, the defect detection model may be a model pre-trained on existing sealing nail weld image training samples, i.e., a source domain model. Furthermore, when applying the defect detection model to the target image set, the parameters of the defect detection model itself may no longer need to be adjusted.

[0058] In addition, the final target detection result may include at least one of the following: whether there is a welding defect in the sealing pin shown in the target image, the probability of the existence of a sealing pin welding defect in each pixel point in the target image, and the regional location of the sealing pin welding defect determined to exist in the target image, etc.

[0059] For example, as shown in FIG3 , after obtaining the target prompt image 33 , the target prompt image 33 can be input into the defect detection model 34 , and the target prompt image 33 can be subjected to sealing nail welding defect detection using the defect detection model 34 , and then the target detection result 35 can be output.

[0060] Based on this, in order to make the defect detection model continuously adapt to the target domain image set, the teacher-student framework can be used to gradually update the visual domain prompt information. In some embodiments, before the above S120, as shown in Figure 4, the sealing nail welding defect detection method provided in the embodiment of the present application can also include the following steps S410-S470:

[0061] S410. Acquire a first image from the target domain image set;

[0062] S420. Process the first image according to the first visual domain prompt information to obtain a first prompt image, where the first image includes any image in the at least one image;

[0063] S430. Using the defect detection model to perform sealing nail welding defect detection on the first prompt image to obtain a first prediction result;

[0064] S440. Smoothing the first visual domain prompt information to obtain second visual domain prompt information;

[0065] S450. Processing the second image according to the second visual domain prompt information to obtain a second prompt image, where the second image is determined based on the first image;

[0066] S460. Using the defect detection model to perform sealing nail welding defect detection on the second prompt image to obtain a second prediction result;

[0067] S470. Based on the difference between the first prediction result and the second prediction result, update the first visual domain prompt information to obtain the target visual domain prompt information.

[0068] Here, the first visual domain cue information may be the original visual domain cue information before being updated during any training process to update the target visual domain cue information. The first image may be any image selected from a set of target domain images, i.e., an image used during the training process to update the target visual domain cue information.

[0069] Exemplarily, the parameter matrix of the first visual domain prompt information and the feature matrix corresponding to the first image can be added point by point to obtain a first prompt image, and the first prompt image can be input into the above-mentioned defect detection model. The defect detection model is used to perform defect detection on the first prompt image, and then a first prediction result can be output. The first prediction result may, for example, include the probability of the existence of a sealing nail welding defect in each pixel point in the first image, and the regional location of the sealing nail welding defect determined to exist in the first image, etc.

[0070] In addition, the second visual domain prompt information may be information obtained by optimizing the first visual domain prompt information, such as smoothing. The second image may be the first image, or another image generated based on the first image.

[0071] Exemplarily, the parameter matrix of the second visual domain prompt information and the feature matrix corresponding to the second image can be added point by point to obtain a second prompt image, and the second prompt image can be input into the above-mentioned defect detection model. The defect detection model is used to perform defect detection on the second prompt image, and then a second prediction result can be output. The second prediction result may, for example, include the probability of a welding defect in the sealing nail at each pixel point in the second image, and the regional location of the welding defect determined to exist in the second image.

[0072] Since the second visual domain prompt information is better than the first visual domain prompt information, the process of using the first visual domain prompt information to process the first image and obtaining the first prediction result after model detection can be used as a student, and the process of using the second visual domain prompt information to process the second image and obtaining the second prediction result after model detection can be used as a teacher to realize the unsupervised training and updating process of the target visual domain prompt information.

[0073] For example, the second prediction result can be used as a label, and the parameters in the first visual domain cue information can be updated based on the difference between the first and second prediction results. After multiple unsupervised training sessions according to the above process, visual domain cue information corresponding to the target domain image set can be obtained upon completion of the training, i.e., the target visual domain cue information.

[0074] In this way, by adopting a teacher-student framework, the target visual domain prompt information is gradually updated in an unsupervised training manner, thereby achieving continuous adaptation to the new target domain image set. In addition, since a large number of sealing nail welding images obtained in real industrial scenarios are unlabeled, and the image labeling process is time-consuming, labor-intensive, and costly, the unsupervised training method adopted in the embodiments of the present application can also reduce the data labeling process, saving time, human and material resources, and reducing costs.

[0075] Based on this, in order to improve the optimization effect of the target visual domain prompt information, in the embodiments of the present application, when smoothing the first visual domain prompt information to determine the second visual domain prompt information, an exponential moving average algorithm can be used for smoothing. In some embodiments, the above-mentioned smoothing of the first visual domain prompt information to obtain the second visual domain prompt information can specifically include:

[0076] Obtain target visual domain prompt information obtained through historical updates, and obtain historical visual domain prompt information;

[0077] Based on the historical visual domain prompt information, the first visual domain prompt information is smoothed according to the exponential moving average algorithm to obtain the second visual domain prompt information.

[0078] To obtain more accurate target visual domain prompt information, embodiments of the present application can update the target visual domain prompt information multiple times for the target domain image set. Each update can generate a new target visual domain prompt information, which will be used as the first visual domain prompt information for the next update. In addition, each update can record the corresponding update time to facilitate subsequent acquisition of historical visual domain prompt information in a chronological order.

[0079] For example, the second visual domain prompt information may be calculated using the following formula (1) according to an exponential moving average algorithm.

[0080] Φ′ t =α·Φ′ t-1 +(1-α)·Φ t (1)

[0081] Among them, Φ′ t is the second visual domain prompt information used in the update process at time t; Φ t is the first visual domain prompt information used in the update process at time t, that is, the historical visual domain prompt information obtained after the update at time t-1 before time t; Φ′ t-1is the second visual domain prompt information used in the update process at time t-1; α is the reference weight, which can be set according to the actual scene requirements.

[0082] In this way, the first visual domain prompt information is smoothed by the above-mentioned exponential moving average algorithm to obtain the second visual domain prompt information, which can make the optimization effect of the second visual domain prompt information more obvious, and then a more accurate second detection result can be obtained based on the second visual domain prompt information, thereby improving the training and updating efficiency of the target visual domain prompt information.

[0083] In addition, in order to further improve the accuracy of the second detection result, in some embodiments of the present application, before processing the second image according to the second visual field prompt information, the sealing nail welding defect detection method provided in the embodiment of the present application may further include:

[0084] Perform enhancement processing on the first image to obtain a second image.

[0085] Here, enhancement processing may include modifying the brightness and contrast of the first image to obtain more image data similar to the first image, such as the second image. Based on this, using the second visual domain cue information to process the second image similar to the first image can improve the robustness of the target visual domain cue information obtained in subsequent updates.

[0086] In addition, when the target visual domain prompt information includes target domain-specific prompt information and target domain-independent prompt information, in some embodiments of the present application, as shown in FIG5 , the above step S470 updates the first visual domain prompt information based on the difference between the first prediction result and the second prediction result to obtain the target visual domain prompt information, which may specifically include the following steps S510-S530:

[0087] S510. Determine a domain-specific prompt loss value and a domain-independent prompt loss value based on the first prediction result and the second prediction result;

[0088] S520. Update the domain-specific prompt information in the first visual domain prompt information according to the domain-specific prompt loss value to obtain target domain-specific prompt information;

[0089] S530. Update the domain-independent prompt information in the first visual domain prompt information according to the domain-independent prompt loss value to obtain target domain-independent prompt information.

[0090] Here, a preset loss function may be used to calculate a domain-specific hint loss value and a domain-independent hint loss value based on the first prediction result and the second prediction result. The preset loss function may be used to measure the difference between the probability distribution in the first prediction result and the probability distribution in the second prediction result as a label. The preset loss function may include, for example, a cross-entropy loss function.

[0091] For example, the domain-specific prompt loss value may be calculated using the following formula (2).

[0092]

[0093] Among them, ω φ Indicates domain-specific prompt information. represents the domain-specific hint loss value, represents the first image, represents the second image obtained by performing random enhancement processing on the first image, represents the student network, represents the teacher network.

[0094] After obtaining the domain-specific cue loss value, the parameters in the parameter matrix corresponding to the domain-specific cue information in the first visual domain cue information can be adjusted based on the domain-specific cue loss value to update the domain-specific cue information. After multiple adjustments and updates, the target domain-specific cue information suitable for the target domain image set can be obtained.

[0095] In addition, for the domain-independent hint loss value corresponding to the domain-independent hint information, additional loss values ​​can be further added on the basis of the above process to suppress domain-sensitive parameters, thereby helping to extract domain-independent knowledge, that is, knowledge that is not specific to any particular domain and can be generalized to different domains.

[0096] After obtaining the domain-independent cue loss value, the parameters in the parameter matrix corresponding to the domain-independent cue information in the first visual domain cue information can be adjusted based on the domain-independent cue loss value to update the domain-independent cue information. After multiple adjustments and updates, the target domain-independent cue information suitable for the target domain image set can be obtained.

[0097] In this way, by using both domain-specific and domain-independent cues, we can effectively adapt to the ever-changing target domain image collection. The domain-specific cues help capture domain-specific knowledge, while the domain-independent cues help preserve domain-shared knowledge, reducing the risks of overfitting, catastrophic forgetting, and error accumulation that may arise during the adaptation process.

[0098] Based on the above implementations, in order to better describe the entire training update process, an example is given below with reference to FIG6 .

[0099] For example, as shown in Figure 6, during a training update process, for the student network in the teacher-student framework, any first image 611 taken from the target domain image set as training data can be point-by-point added to the domain-specific and domain-independent prompt information in the first visual domain prompt information 612. The coordinates of the parameters in the domain-specific and domain-independent prompt information are then applied to the first image 613, thereby creating a first prompt image 613. The first prompt image 613 is input into the defect detection model 630, and a first prediction result 614 is output.

[0100] Additionally, data enhancement can be performed on the first image 611 to obtain a second image 621, and exponential moving average processing can be performed on the domain-specific and domain-independent cues in the first visual domain cues 612 to obtain second visual domain cues 622. For the teacher network in the teacher-student framework, the second image 621 can be added point by point to the domain-specific and domain-independent cues in the second visual domain cues 622. The coordinates of the parameters in the domain-specific and domain-independent cues are then applied to the second image 621 to create a second cued image 623. The second cued image 623 is input into the defect detection model 630 to output a second prediction result 624.

[0101] Based on the first prediction result 614 and the second prediction result 624, a domain-specific prompt loss value L can be determined. dsp The domain-independent prompt loss value L dap , using the domain-specific prompt loss value L dsp The domain-specific prompt information in the first visual domain prompt information 612 can be updated, and the domain-independent prompt loss value L can be used. dap The domain-independent prompt information in the first visual domain prompt information 612 may be updated. In this way, a training update process is completed.

[0102] Based on this, in order to suppress the role of domain-sensitive parameters in domain-independent prompt information, an additional steady-state regularization loss value can be added when calculating the domain-independent prompt loss value. In some embodiments of the present application, as shown in Figure 7, the above step S510 determines the domain-independent prompt loss value based on the first prediction result and the second prediction result, and specifically may include the following steps S710-S740:

[0103] S710. Determine a first loss value based on the first prediction result and the second prediction result;

[0104] S720. Obtain the domain-specific prompt loss value and the domain-independent prompt loss value corresponding to the historical domain image set to obtain the historical visual domain prompt loss value;

[0105] S730. Determine a steady-state regularization loss value corresponding to the target domain image set based on the historical visual domain prompt loss value;

[0106] S740. Determine a domain-independent hint loss value based on the first loss value and the steady-state regularization loss value.

[0107] Here, the method for determining the first loss value can be the same as the method for determining the above-mentioned domain-specific prompt loss value, that is, for example, the cross-entropy loss function shown in the above formula (2) can be used to calculate the first loss value based on the first prediction result and the second prediction result as part of the domain-independent prompt loss value.

[0108] In addition, the historical domain image set may include, for example, various domain image sets to which the model has been adapted before the target domain image set.

[0109] Exemplarily, for each domain image set that the model has adapted to before the target domain image set, the sum of the domain-specific prompt loss value and the domain-independent prompt loss value corresponding to each domain image set at a certain training update time can be obtained as the historical visual domain prompt loss value corresponding to the domain image set, and then the historical visual domain prompt loss value corresponding to each domain image set is used to determine the domain-sensitive parameters, and then the steady-state regularization loss value corresponding to the target domain image set is calculated. The steady-state regularization loss value is used to penalize these sensitive parameters, and the domain-independent parameters are stably updated to consolidate domain-independent knowledge.

[0110] In some examples, after the first loss value and the steady-state regularization loss value are calculated, the domain-independent hint loss value can be calculated according to the following formula (3).

[0111]

[0112] Among them, ψ δ Indicates domain-specific prompt information. represents the domain-independent hint loss value, L(ψ δ ) represents the steady-state regularization loss value, represents the first image, represents the second image obtained by performing random enhancement processing on the first image, represents the student network, represents the teacher network.

[0113] In this way, by using an additional steady-state regularization loss value in the process of updating domain-independent hint information, the steady-state regularization mechanism can be utilized to limit the sensitivity of parameters in the domain-independent hint information to domain changes, ensuring that the domain-independent hint information does not over-adapt to any single domain, but instead captures more general and transferable knowledge, thereby promoting the domain-independent hint information to learn more general knowledge.

[0114] Based on this, in some embodiments, as shown in FIG8 , the above step S730 determines the steady-state regularization loss value corresponding to the target domain image set based on the historical visual domain cue loss value, and specifically may include the following steps S810-S830:

[0115] S810. Determine the parameter importance of each parameter in the visual domain prompt information corresponding to the historical domain image set according to the historical visual domain prompt loss value;

[0116] S820. Determine a steady-state factor corresponding to each parameter in the target visual domain prompt information based on the parameter importance of each parameter in the visual domain prompt information corresponding to the historical domain image set and the difference value of each parameter at different times;

[0117] S830. Determine a steady-state regularization loss value of the target visual domain prompt information according to the steady-state factor.

[0118] For example, in order to measure the sensitivity of each parameter in the visual domain cue information to changes in different domains, the following gradient formula (4) can be defined.

[0119]

[0120] Among them, θ(t) represents the parameters in the visual domain cue information at time t, g(θ(t)) represents the gradient value corresponding to the parameters in the visual domain cue information at time t, and L represents the total loss value obtained during the updated training process at time t, that is, the sum of the domain-specific cue loss value and the domain-independent cue loss value.

[0121] Based on this, parameter importance can be used to quantify the contribution of each parameter to the total loss value. For example, the parameter importance of each parameter can be calculated using the following formula (5).

[0122]

[0123]

[0124]

[0125] in, represents the parameter importance of the i-th parameter in the historical domain image set ν. t1 and t0 are the time points of parameter update in the historical domain image set ν and its adjacent previous historical domain image set, respectively.

[0126] In addition, the difference values ​​of the parameters in the historical domain image set ν at different update time points can also be calculated according to the following formula (6).

[0127]

[0128] in, represents the difference between the i-th parameter in the historical domain image set ν at the update time point t and t-1, represents the value of the i-th parameter in the historical domain image set ν at the update time point t, Represents the value of the i-th parameter in the historical domain image set ν at the update time point t-1.

[0129] Based on this, the importance of the parameter corresponding to the i-th parameter in the above historical field image set ν can be calculated as follows: And the difference between different update time points Calculate the steady-state factor corresponding to the i-th parameter. For example, the steady-state factor can be calculated according to the following formula (7).

[0130]

[0131] Among them, τ represents the current target domain image set, ν represents each domain image set before the target domain image set τ, Indicates the steady-state factor corresponding to the i-th parameter in the visual domain prompt information corresponding to the target domain image set. In this case, ξ can be introduced into the steady-state factor, where the value of ξ can be, for example, 0.01.

[0132] From the above formula (7), we can see that Smaller means that the value of the parameter changes slightly. When is larger, the contribution of the parameter to the total loss value is more significant. Relatively small or When is relatively large, the i-th parameter can be considered to be sensitive to changes in the domain.

[0133] Based on this, the steady-state regularization loss value can be calculated according to the following formula (8).

[0134]

[0135] Among them, β is used to control the contribution of the steady-state regularization loss value to the domain-independent prompt loss value, and its value can be set according to the needs of the actual application scenario. θ represents the parameter in the visual domain prompt information corresponding to the current target domain image set, and θ * Represents the parameters in the visual domain prompt information corresponding to the previous target domain image set before the target domain image set.

[0136] In this way, the steady-state regularization loss value can be calculated through the above process to penalize the update of sensitive parameters, prevent domain-independent prompt information from over-adapting to a specific field, and thus promote its learning of more general knowledge.

[0137] In addition, it is important to note that when entering a new field, will be updated, and It is also set to 0, thereby updating the steady-state regularization loss value. Therefore, based on the prediction confidence of the detection results output by the model, it can be determined whether the current target domain image set has undergone domain shift, that is, whether the sealing nail welding image to be detected already belongs to the new domain image set.

[0138] Based on this, in some embodiments, as shown in FIG9 , after the above S130 , the sealing pin welding defect detection method provided in the embodiment of the present application may further include the following steps S140 - S150 :

[0139] S140. Obtain the target prediction confidence of the defect detection model output for the target detection result;

[0140] S150. When the difference between the target prediction confidence and the historical prediction confidence is greater than a preset threshold, update the steady-state regularization loss value.

[0141] Here, the defect detection model can output the corresponding prediction confidence level each time it outputs a detection result. Based on this, after the defect detection model completes the detection of the target image, it can also output the target prediction confidence level for that target detection result at the same time as the target detection result.

[0142] For example, the difference between the target prediction confidence and the historical prediction confidence can be calculated according to the following formula (9).

[0143] ΔConf=Conf(t)-Conf(t-1) (9)

[0144] Where Conf(t) represents the target prediction confidence, Conf(t-1) represents the prediction confidence of the last image to be detected before the model outputs the target prediction confidence, that is, the historical prediction confidence, and ΔConf represents the difference between the target prediction confidence and the historical prediction confidence. When ΔConf is greater than the preset threshold S, it can be determined that the target domain image set has undergone domain shift. At this time, and will be updated, thereby causing the steady-state regularization loss value to be updated.

[0145] In this way, through the above-mentioned domain shift detection process, the model can automatically detect new domains and then continuously adapt to the new domain image set, further improving the model's automatic generalization ability.

[0146] It should be noted that the application scenarios described in the above-mentioned embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Ordinary technicians in this field can know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0147] Based on the same inventive concept, the present application also provides a sealing pin welding defect detection device, which is described in detail with reference to FIG10 .

[0148] FIG10 is a schematic structural diagram of a sealing pin welding defect detection device provided in one embodiment of the present application.

[0149] As shown in FIG10 , the sealing pin welding defect detection device 1000 may include:

[0150] An image acquisition module 1001 is configured to acquire a target image to be processed, wherein the target image includes a sealing nail welding image acquired for a sealing nail welded using a target welding type;

[0151] An image processing module 1002 is configured to process the target image according to target visual domain prompt information to obtain a target prompt image, wherein the target visual domain prompt information includes visual domain prompt information determined based on a target domain image set, wherein the target domain image set includes a plurality of sealing nail welding images corresponding to the target welding type;

[0152] The defect detection module 1003 is used to use a defect detection model to perform sealing nail welding defect detection on the target prompt image to obtain a target detection result, wherein the defect detection model includes a model pre-trained using an image set in other fields other than the target field image set.

[0153] The sealing nail welding defect detection device 1000 is described in detail below, as shown below:

[0154] In some embodiments, the sealing pin welding defect detection device 1000 further includes:

[0155] A first acquisition module, configured to acquire a first image from the target domain image set;

[0156] a first processing module, configured to process a first image according to the first visual domain prompt information to obtain a first prompt image, where the first image includes any image of the at least one image;

[0157] A first detection module, configured to perform sealing nail welding defect detection on the first prompt image using the defect detection model to obtain a first prediction result;

[0158] a smoothing processing module, configured to perform smoothing on the first visual domain prompt information to obtain second visual domain prompt information;

[0159] a second processing module, configured to process a second image according to the second visual domain prompt information to obtain a second prompt image, where the second image is determined based on the first image;

[0160] A second detection module is configured to perform sealing nail welding defect detection on the second prompt image using the defect detection model to obtain a second prediction result;

[0161] The first updating module is configured to update the first visual domain prompt information based on a difference between the first prediction result and the second prediction result to obtain target visual domain prompt information.

[0162] In some embodiments, the smoothing processing module includes:

[0163] The first acquisition submodule is used to obtain the target visual domain prompt information obtained through historical updates, and obtain the historical visual domain prompt information;

[0164] The first processing submodule is configured to perform a smoothing process on the first visual domain prompt information according to an exponential moving average algorithm based on the historical visual domain prompt information to obtain the second visual domain prompt information.

[0165] In some embodiments, the smoothing processing module further includes:

[0166] The second processing submodule is configured to perform enhancement processing on the first image to obtain the second image before processing the second image according to the second visual domain prompt information.

[0167] In some embodiments, the target visual domain prompt information includes target domain specific prompt information and target domain irrelevant prompt information.

[0168] Based on this, the first update module includes:

[0169] a first determining submodule, configured to determine a domain-specific prompt loss value and a domain-independent prompt loss value based on the first prediction result and the second prediction result;

[0170] a first updating submodule, configured to update the domain-specific prompt information in the first visual domain prompt information according to the domain-specific prompt loss value to obtain target domain-specific prompt information;

[0171] The second updating submodule is configured to update the domain-independent prompt information in the first visual domain prompt information according to the domain-independent prompt loss value to obtain target domain-independent prompt information.

[0172] In some embodiments, the first determining submodule includes:

[0173] A first determining unit, configured to determine a first loss value based on the first prediction result and the second prediction result;

[0174] A first acquisition unit is configured to acquire a domain-specific cue loss value and a domain-independent cue loss value corresponding to a historical domain image set to obtain a historical visual domain cue loss value;

[0175] a second determining unit, configured to determine a steady-state regularization loss value corresponding to the target domain image set according to the historical visual domain cue loss value;

[0176] The third determining unit is configured to determine a domain-independent hint loss value according to the first loss value and the steady-state regularization loss value.

[0177] In some embodiments, the second determining unit includes:

[0178] A first determining subunit is configured to determine the parameter importance of each parameter in the visual domain cue information corresponding to the historical domain image set according to the historical visual domain cue loss value;

[0179] a second determining subunit, configured to determine a steady-state factor corresponding to each parameter in the target visual domain prompt information based on the parameter importance of each parameter in the visual domain prompt information corresponding to the historical domain image set and the difference value of each parameter at different times;

[0180] The third determining subunit is configured to determine a steady-state regularization loss value of the target visual domain prompt information according to the steady-state factor.

[0181] In some embodiments, the sealing pin welding defect detection device 1000 further includes:

[0182] a confidence acquisition module, configured to, after performing sealing nail welding defect detection on the target prompt image using a defect detection model and obtaining a target detection result, acquire a target prediction confidence output by the defect detection model for the target detection result;

[0183] A loss updating module is used to update the steady-state regularization loss value when the difference between the target prediction confidence and the historical prediction confidence is greater than a preset threshold.

[0184] Thus, by using the sealing nail welding images collected for sealing nails welded using the target welding type in the target domain image set, the target visual domain prompt information corresponding to the target domain image set is determined, and then when the defect detection model pre-trained by the other domain image set is used to perform defect detection on the target image corresponding to the target welding type, the target image can be first processed according to the target visual domain prompt information, and then the defect detection model can be used to perform welding defect detection on the processed target prompt image. In this way, since the image-level visual domain prompt for the target welding type is used when the pre-trained defect detection model is used to perform defect detection on the sealing nail welding image of the new target welding type, the generalization of the model in the target welding type sealing nail welding image defect detection can be improved, and the model detection accuracy can be improved.

[0185] FIG11 is a schematic structural diagram of an electronic device provided in one embodiment of the present application.

[0186] The electronic device 1100 may include a processor 1101 and a memory 1102 storing computer program instructions.

[0187] Specifically, the processor 1101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0188] Memory 1102 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 1102 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1102 may include removable or non-removable (or fixed) media. Where appropriate, memory 1102 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, memory 1102 is a non-volatile solid-state memory.

[0189] In certain embodiments, the memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present application.

[0190] The processor 1101 reads and executes computer program instructions stored in the memory 1102 to implement any one of the sealing pin welding defect detection methods in the above embodiments.

[0191] In some examples, the electronic device 1100 may further include a communication interface 1103 and a bus 1110. As shown in FIG11 , the processor 1101, the memory 1102, and the communication interface 1103 are connected via the bus 1110 and communicate with each other.

[0192] The communication interface 1103 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0193] Bus 1110 includes hardware, software or both, and the components of online data flow metering equipment are coupled to each other. For example, but not limitation, bus 1110 may include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1110 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0194] Illustratively, the electronic device 1100 may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA).

[0195] The electronic device 1100 can execute the sealing pin welding defect detection method in the embodiment of the present application, thereby realizing the sealing pin welding defect detection method and device described in combination with Figures 1 to 10.

[0196] In addition, in combination with the sealing pin welding defect detection method in the above embodiment, the embodiment of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the sealing pin welding defect detection methods in the above embodiment is implemented. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0197] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0198] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0199] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0200] Aspects of the present application have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed via the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. This processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or the flowchart and the combination of the boxes in the block diagram and / or the flowchart can also be implemented by the dedicated hardware that performs the specified function or action, or can be implemented by the combination of dedicated hardware and computer instructions.

[0201] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A method for detecting sealing pin welding defects, comprising: Acquiring a target image to be processed, wherein the target image includes a sealing nail welding image collected for a sealing nail welded using a target welding type; Processing the target image according to target visual domain prompt information to obtain a target prompt image, wherein the target visual domain prompt information includes visual domain prompt information determined based on a target domain image set, and the target domain image set includes a plurality of sealing nail welding images corresponding to the target welding type; The target prompt image is subjected to sealing nail welding defect detection using a defect detection model to obtain a target detection result, wherein the defect detection model includes a model pre-trained using an image set in another domain other than the target domain image set.

2. The method according to claim 1, wherein Before processing the target image according to the target visual domain prompt information to obtain the target prompt image, the method further includes: Acquire a first image from the target domain image set; Processing the first image according to the first visual domain prompt information to obtain a first prompt image; Using the defect detection model to perform sealing nail welding defect detection on the first prompt image to obtain a first prediction result; performing smoothing processing on the first visual domain prompt information to obtain second visual domain prompt information; Processing the second image according to the second visual domain prompt information to obtain a second prompt image, where the second image is determined based on the first image; Using the defect detection model to perform sealing nail welding defect detection on the second prompt image to obtain a second prediction result; Based on the difference between the first prediction result and the second prediction result, the first visual domain prompt information is updated to obtain target visual domain prompt information.

3. The method according to claim 2, wherein: The smoothing process of the first visual domain prompt information to obtain the second visual domain prompt information includes: Obtain target visual domain prompt information obtained through historical updates, and obtain historical visual domain prompt information; Based on the historical visual domain prompt information, the first visual domain prompt information is smoothed according to an exponential moving average algorithm to obtain second visual domain prompt information.

4. The method according to claim 2 or 3, wherein: Before processing the second image according to the second visual domain prompt information, the method further includes: Perform enhancement processing on the first image to obtain the second image.

5. The method according to any one of claims 2 to 4, wherein: The target visual domain prompt information includes target domain specific prompt information and target domain irrelevant prompt information; The updating of the first visual domain prompt information based on the difference between the first prediction result and the second prediction result to obtain the target visual domain prompt information includes: determining a domain-specific hint loss value and a domain-independent hint loss value based on the first prediction result and the second prediction result; updating the domain-specific prompt information in the first visual domain prompt information according to the domain-specific prompt loss value to obtain the target domain-specific prompt information; The domain-independent prompt information in the first visual domain prompt information is updated according to the domain-independent prompt loss value to obtain the target domain-independent prompt information.

6. The method according to claim 5, wherein: Determining a domain-independent hint loss value based on the first prediction result and the second prediction result includes: Determine a first loss value based on the first prediction result and the second prediction result; Obtain the domain-specific cue loss value and the domain-independent cue loss value corresponding to the historical domain image set to obtain the historical visual domain cue loss value; Determining a steady-state regularization loss value corresponding to the target domain image set based on the historical visual domain cue loss value; A domain-independent hint loss value is determined according to the first loss value and the steady-state regularization loss value.

7. The method according to claim 6, wherein: The determining, based on the historical visual domain cue loss value, a steady-state regularization loss value corresponding to the target domain image set comprises: determining, according to the historical visual domain cue loss value, the parameter importance of each parameter in the visual domain cue information corresponding to the historical domain image set; Determining a steady-state factor corresponding to each parameter in the target visual domain prompt information based on the parameter importance of each parameter in the visual domain prompt information corresponding to the historical domain image set and the difference value of each parameter at different times; A steady-state regularization loss value of the target visual domain prompt information is determined according to the steady-state factor.

8. The method according to claim 6 or 7, wherein: After performing sealing nail welding defect detection on the target prompt image using the defect detection model to obtain a target detection result, the method further includes: Obtaining a target prediction confidence level output by the defect detection model for the target detection result; When the difference between the target prediction confidence and the historical prediction confidence is greater than a preset threshold, the steady-state regularization loss value is updated.

9. The method according to any one of claims 1 to 8, wherein: The target visual domain prompt information is a learnable parameter matrix, and the target image is processed according to the target visual domain prompt information to obtain a target prompt image, including: The parameter matrix in the target visual domain prompt information is added point by point to the feature matrix corresponding to the target image to obtain a target prompt image.

10. The method according to any one of claims 2 to 8, wherein: The acquiring of a first image from the target domain image set includes: Acquire at least one image from the target field image set corresponding to the target welding type; Any image among the at least one image is used as the first image.

11. In the method according to any one of claims 2 to 8, updating the first visual domain prompt information based on the difference between the first prediction result and the second prediction result to obtain the target visual domain prompt information comprises: The process of processing the first image using the first visual domain prompt information and obtaining the first prediction result after detection by the defect detection model is used as the student network, and the process of processing the second image using the second visual domain prompt information and obtaining the second prediction result after detection by the defect detection model is used as the teacher network to perform unsupervised training and update the target visual domain prompt information.

12. The method according to claim 3, wherein: The step of smoothing the first visual domain prompt information based on the historical visual domain prompt information according to an exponential moving average algorithm to obtain the second visual domain prompt information includes: The second visual prompt information is calculated using the following formula: F′ t =α·Φ′ t-1 +(1-a)·Φ t Among them, Φ′ t is the second visual domain prompt information used in the update process at time t, Φ t is the first visual domain prompt information used in the update process at time t, that is, the historical visual domain prompt information obtained after the update at time t-1 before time t, Φ′ t-1 is the second visual domain prompt information used in the update process at time t-1, and α is the reference weight, which is set according to the actual scene requirements.

13. The method according to claim 4, wherein: The performing enhancement processing on the first image to obtain the second image includes: At least one of the brightness and contrast of the first image is changed.

14. The method according to claim 5, wherein Determining a domain-specific hint loss value based on the first prediction result and the second prediction result includes: The domain-specific hint loss value is calculated using the following formula: Among them, ω φ Indicates domain-specific prompt information. represents the domain-specific hint loss value, represents the first image, represents the second image obtained by performing random enhancement processing on the first image, represents the student network, represents the teacher network.

15. The method according to claim 6, wherein The determining a first loss value based on the first prediction result and the second prediction result includes: A cross entropy loss function is used to calculate a first loss value according to the first prediction result and the second prediction result.

16. The method according to claim 6, wherein The determining of a domain-independent hint loss value according to the first loss value and the steady-state regularization loss value includes: The domain-independent prompt loss value is calculated using the following formula: Among them, ψ δ Indicates domain-specific prompt information. represents the domain-independent hint loss value, L(ψ δ ) represents the steady-state regularization loss value, represents the first image, represents the second image obtained by performing random enhancement processing on the first image, represents the student network, represents the teacher network.

17. The method according to claim 7, wherein: Determining the parameter importance of each parameter in the visual domain prompt information corresponding to the historical domain image set according to the historical visual domain prompt loss value includes: The parameter importance of each parameter is calculated using the following formula: in, represents the parameter importance of the i-th parameter in the historical domain image set ν, t1 and t0 are the parameter update time points in the historical domain image set ν and its adjacent previous historical domain image set, respectively.

18. A sealing nail welding defect detection device, comprising: An image acquisition module, configured to acquire a target image to be processed, wherein the target image includes a sealing nail welding image collected for a sealing nail welded using a target welding type; an image processing module, configured to process the target image according to target visual domain prompt information to obtain a target prompt image, wherein the target visual domain prompt information includes visual domain prompt information determined based on a target domain image set, wherein the target domain image set includes a plurality of sealing nail welding images corresponding to the target welding type; A defect detection module is used to use a defect detection model to perform sealing nail welding defect detection on the target prompt image to obtain a target detection result, wherein the defect detection model includes a model pre-trained using an image set in other fields other than the target field image set.

19. An electronic device, wherein: include: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the steps of the sealing pin welding defect detection method according to any one of claims 1 to 17 are implemented.

20. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the sealing pin welding defect detection method according to any one of claims 1 to 17.

Images