Welding spot quality defect sample data enhancement method based on generative adversarial network

Abstract

The application relates to the technical field of image generation, in particular to a welding spot quality defect sample data enhancement method based on a generative adversarial network, which comprises the following steps: acquiring a plurality of welding spot images and welding environment parameters corresponding to each welding spot image; establishing a welding process rule library according to the plurality of welding spot images and the welding environment parameters corresponding to each welding spot image; determining a physical law deviation of a synthetic image of the generative adversarial network according to the welding process rule library; training the generative adversarial network according to the physical law deviation and an adversarial loss of the generative adversarial network, so as to obtain a trained sample generation model; and generating a welding spot quality defect sample set according to the sample generation model. The method has high process credibility and stable quality of the synthetic image.

Description

Technical Field

[0001] This application relates to the field of image generation technology, specifically to a method for enhancing sample data of weld joint quality defects based on generative adversarial networks. Background Technology

[0002] Welding, a critical joining process in modern manufacturing, is widely used in high-end equipment manufacturing fields such as automobiles, aerospace, rail transportation, and electronic packaging. The quality of weld joints directly determines the structural integrity and service reliability of products. However, due to factors such as process fluctuations, environmental interference, and material differences, typical defects such as porosity, cracks, incomplete penetration, lack of fusion, and slag inclusions are inevitably generated during the welding process. In recent years, deep learning-based defect detection methods, with their powerful automatic feature extraction and pattern recognition capabilities, have gradually replaced traditional manual visual inspection and rule-based algorithms, becoming the mainstream technology for industrial visual inspection. However, the performance of deep learning models is highly dependent on large-scale, high-quality, and balanced labeled datasets covering various types of defects. In actual industrial scenarios, weld joint defects exhibit a typical long-tail distribution, with a severe shortage of rare defect samples (such as cracks and lack of fusion), making it difficult for defect detection models to accurately identify rare defects.

[0003] Currently, data augmentation methods based on Generative Adversarial Networks (GANs) can be used to increase the number of rare defect samples. GANs can learn the features of real defect images through adversarial training between the generator and the discriminator. By adjusting the input variables, new defect images can be generated, thereby expanding the training dataset and improving the defect detection model's ability to identify various types of defects.

[0004] However, when traditional generative adversarial networks generate defect images, they usually treat environmental features such as lighting, material, and viewing angle as independent variables and adjust them separately. They mainly optimize the visual realism of the generated defect images, but lack explicit constraints on the physical laws of welding processes. This can lead to the generated defect images having features that are "impossible in terms of process", such as excessively narrow weld width under high current or excessively wide weld width under high welding speed. The quality stability of data augmentation is also poor, which makes it difficult for the training effect and generalization performance of the defect detection model to meet the needs of actual production lines. Summary of the Invention

[0005] To address the technical problem that defect images generated by traditional generative adversarial networks may exhibit features that are "technically impossible," this application aims to provide a method for enhancing solder joint quality defect sample data based on generative adversarial networks. The specific technical solution adopted is as follows:

[0006] This application provides a method for enhancing weld joint quality defect sample data based on generative adversarial networks (GANs), comprising: acquiring multiple weld joint images and welding environment parameters corresponding to each weld joint image; establishing a welding process rule base based on the multiple weld joint images and the welding environment parameters corresponding to each weld joint image, wherein the welding process rule base includes multiple sets of mapping relationships between welding environment parameters and image physical features, wherein the image physical features are used to characterize the geometric shape and defect distribution attributes in the weld joint images; determining the physical law deviation of the synthesized images of the GAN based on the welding process rule base; training the GAN based on the physical law deviation and the adversarial loss of the GAN to obtain a trained sample generation model; and generating a weld joint quality defect sample set based on the sample generation model.

[0007] Optionally, the above-mentioned establishment of a welding process rule base based on multiple weld point images and welding environment parameters corresponding to each weld point image includes: determining the image physical features corresponding to each weld point image, wherein the image physical features include at least weld width and defect morphology complexity; and establishing multiple sets of mapping relationships based on the welding environment parameters and image physical features corresponding to each weld point image through regression analysis.

[0008] Optionally, the above-mentioned solder joint image also includes the outline perimeter and area of ​​the defect region. The above-mentioned determination of the image physical features corresponding to each solder joint image includes: performing edge detection on each solder joint image, determining the weld width based on the edge detection results and camera calibration parameters; and determining the morphological complexity of the defect based on the outline perimeter and area of ​​the defect region.

[0009] Optionally, the above-mentioned determination of the physical law deviation of the synthesized image based on the welding process rule base includes: extracting the actual image physical features of the synthesized image; querying the welding process rule base according to the target welding environment parameters used when generating the synthesized image to obtain the predicted image physical features; and determining the physical law deviation based on the deviation between the actual image physical features and the predicted image physical features.

[0010] Optionally, the above-mentioned training of the generative adversarial network based on the physical law deviation and the adversarial loss of the generative adversarial network to obtain the trained sample generation model includes: in each training iteration, obtaining the synthesized image of the generative adversarial network and the adversarial loss of the generative adversarial network in the current iteration; determining the total loss based on the physical law deviation and adversarial loss of the synthesized image; and updating the network parameters of the generative adversarial network based on the total loss.

[0011] Optionally, determining the total loss based on the physical law deviation and adversarial loss of the synthesized image includes: inputting the synthesized image into a parameter prediction network to obtain predicted values ​​of welding environment parameters; determining the mapping error based on the deviation between the predicted values ​​of welding environment parameters and the target welding environment parameters; and determining the total loss based on the physical law deviation, mapping error, and adversarial loss of the synthesized image.

[0012] Optionally, determining the total loss based on the physical law deviation, mapping error, and adversarial loss of the synthesized image includes: splitting the latent space of the generative adversarial network into an environmental factor latent space and a defect content latent space; transforming the input variables of the environmental factor latent space and the defect content latent space into environmental feature vectors and defect feature vectors; determining the mutual information between each dimension within the environmental feature vector, between each dimension within the defect feature vector, and between the environmental feature vector and the defect feature vector; determining the average value of all mutual information as a regularization term; and weighted summing the physical law deviation, mapping error, regularization term, and adversarial loss of the synthesized image to obtain the total loss.

[0013] Optionally, the method further includes: fixing the values ​​of other latent space dimensions, traversing the value range of the target dimension, generating multiple sets of images, wherein the target dimension is any one of all latent space dimensions, and the other latent space dimensions are dimensions other than the target dimension among all latent space dimensions; measuring the factor values ​​corresponding to the target dimension in the multiple sets of images; determining the correlation between the value sequence of the target dimension and the factor value sequence; determining the decoupling score based on the mean absolute value of the correlation of all latent space dimensions; and adjusting the weight of the regularization term based on the decoupling score until the decoupling score meets the judgment condition.

[0014] Optionally, after obtaining the solder joint quality defect sample set, the method further includes: performing a quality inspection on the solder joint quality defect sample set; when the solder joint quality defect sample set passes the quality inspection, the solder joint quality defect sample set is determined as a usable sample set.

[0015] Optionally, the above-mentioned quality inspection of the solder joint quality defect sample set includes: determining a distribution difference index based on the solder joint quality defect sample set and the real sample set; determining a defect coverage rate based on the number of defect types covered in the solder joint quality defect sample set; and confirming that the solder joint quality defect sample set passes the quality inspection when the distribution difference index meets the preset distribution conditions and the defect coverage rate meets the preset coverage conditions.

[0016] This application has the following beneficial effects:

[0017] By acquiring weld point images and corresponding welding environment parameters, and establishing a welding process rule base, the overall mapping law between welding environment parameters and the physical features (geometric shape and defect distribution attributes) of weld point images is quantified. This provides a queryable and calculable evaluation standard for the physical authenticity of synthesized images. During training, the physical law deviation of synthesized images is determined in real time according to the welding process rule base, enabling the generator to perceive in real time whether its output violates the physical laws of welding. The physical law deviation and the adversarial loss of the generative adversarial network are used as training objectives, so that the generator must meet the physical law constraints while improving visual authenticity, ensuring that the synthesized images still have process credibility under multi-factor coupling conditions. Finally, the weld point quality defect sample set generated by the trained sample generation model not only has the high diversity brought by the generative adversarial network, but also its geometric features strictly correspond to the input welding environment parameters, and the quality stability is high. It can be directly used to supplement the training data of rare defects (such as cracks and lack of fusion) in the long-tail distribution, thereby improving the training effect and generalization of the defect detection model. Attached Figure Description

[0018] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A method flow for enhancing solder joint quality defect sample data based on generative adversarial networks, provided in one embodiment of this application. Figure 1 ;

[0020] Figure 2 A method flow for enhancing solder joint quality defect sample data based on generative adversarial networks, provided in one embodiment of this application. Figure 2 . Detailed Implementation

[0021] To further illustrate the technical means and effects adopted by this application to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method for enhancing solder joint quality defect sample data based on generative adversarial networks proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0023] The following description, in conjunction with the accompanying drawings, details a specific scheme for a method to enhance sample data of solder joint quality defects based on generative adversarial networks, as provided in this application.

[0024] Please see Figure 1 The diagram illustrates a flowchart of a method for enhancing sample data of solder joint quality defects based on generative adversarial networks, according to an embodiment of this application.

[0025] like Figure 1 As shown, the method for enhancing solder joint quality defect sample data based on generative adversarial networks includes S101-S105.

[0026] S101. Obtain multiple solder joint images and the corresponding welding environment parameters for each solder joint image.

[0027] It should be understood that a solder joint image refers to a true grayscale image of the solder joint surface taken after the soldering is completed, while the soldering environment parameters refer to the parameters used when soldering the solder joint.

[0028] Alternatively, images of solder joints can be acquired using an industrial area scan camera (e.g., a 20-megapixel industrial area scan camera with a frame rate of 30fps) paired with a ring fill light (e.g., a ring fill light with a color temperature of 5500K).

[0029] Optionally, the acquired solder joint images can be quality-screened to remove blurry, overexposed, or underexposed images, and the defect locations, defect areas (including the outline and perimeter of the defect area), and defect types in the retained solder joint images can be manually labeled. The defects in these multiple solder joint images should at least include porosity, cracks, incomplete penetration, lack of fusion, slag inclusions, etc., with 100-500 images collected for each type of defect (100 images for rare defects and 500 images for common defects).

[0030] Optionally, weld seam recognition can also be performed on the weld seam image, for example, by manually annotating or using image segmentation algorithms to determine the weld seam direction and the center position of the weld seam.

[0031] Welding environment parameters include at least current, voltage, and welding speed.

[0032] Optionally, a welding parameter acquisition instrument (current accuracy ±0.1A, voltage accuracy ±0.05V, welding speed accuracy ±0.1cm / min) can be used to record the welding environment parameters of the weld point. Then, each type of welding environment parameter can be normalized. For example, based on the maximum and minimum values ​​of each type of welding environment parameter collected, a maximum-minimum normalization can be performed to map current, voltage, and welding speed of different dimensions to a unified numerical range.

[0033] After obtaining multiple weld point images and welding environment parameters, the weld point images and welding environment parameters can be associated one-to-one using timestamps to establish a correspondence between images and parameters, forming a multimodal dataset. This multimodal dataset can be divided into a training set, a validation set, and a test set according to a preset ratio (e.g., 7:2:1), which are used for model training, parameter tuning, and performance verification, respectively.

[0034] S102. Establish a welding process rule library based on multiple weld point images and the welding environment parameters corresponding to each weld point image.

[0035] The welding process rule library includes the mapping relationship between welding environment parameters and image physical features. The image physical features are used to characterize the geometric shape and defect distribution attributes in the weld point image.

[0036] In one alternative implementation, the physical features of each weld point image can be determined separately, and multiple sets of mapping relationships can be established through regression analysis based on the welding environment parameters and physical features of each weld point image.

[0037] The physical features of the image include at least weld width and defect morphology complexity. Weld width refers to the width of the weld joint fusion zone in the weld joint image, and defect morphology complexity is used to characterize the degree to which the defect contour deviates from a regular circle.

[0038] It should be understood that in a weld joint image, the regions, arranged spatially from the outside in, are the base material region, the heat-affected zone (HAZ), and the weld fusion zone. The base material region is the original material region unaffected by welding heat; the HAZ is a transitional region that is heated but not melted, located between the base material region and the weld fusion zone; the weld fusion zone is the metal region that completely melts and then resolidifies during welding, and is the core region of the weld. Defective regions are typically abnormal areas (such as porosity, cracks, or lack of fusion) that appear inside or near the weld fusion zone.

[0039] Optionally, edge detection can be performed on each weld point image to obtain multiple contours. The minimum circumcircle of each contour is calculated. Contours whose distance from the center of the minimum circumcircle to the center of the weld point is less than 20% of the weld point radius are identified as the fusion zone boundary. Hough line detection is performed on the fusion zone boundary to obtain multiple candidate edge lines. Using the geometric center of the weld fusion zone as a reference, the relative lateral position of each candidate edge line is determined (i.e., relative to the geometric center of the weld fusion zone, on which side of the geometric center in the direction perpendicular to the weld direction). Based on the relative lateral position of the candidate edge lines, the multiple candidate edge lines are divided into weld fusion zones. The first set of side edge lines and the second set of side edge lines on both sides of the fusion zone are defined. Then, the candidate edge line closest to the outside of the fusion zone is selected from the first set of side edge lines as the first outermost edge line, and the candidate edge line closest to the outside of the fusion zone is selected from the second set of side edge lines as the second outermost edge line. The vertical distance between the first outermost edge line and the second outermost edge line is calculated and determined as the fusion width value in pixels. Then, according to the pre-calibrated camera parameters, i.e., pixel equivalent (in millimeters per pixel), the pixel-unit fusion width value is converted into the fusion width value of the actual physical distance, resulting in the fusion width in millimeters.

[0040] Alternatively, the edge detection algorithm can be the Canny edge detection algorithm.

[0041] Optionally, the solder joint image also includes the perimeter and area of ​​the defect region, and the morphological complexity of the defect is determined based on the perimeter and area of ​​the defect region.

[0042] For example, the morphological complexity of a defect can be calculated using the following formula:

[0043]

[0044] in, Image of solder joint The complexity of the defect form, Image of solder joint The area of ​​the defect region, This indicates the perimeter of the defective area.

[0045] In this formula, Indicates roundness. The closer the value is to 1, the closer the defect area is to a circle, and the lower the complexity of the defect shape. The closer to 0, the more elongated or irregular the defect area, and the higher the complexity of the defect morphology.

[0046] It should be noted that when the perimeter of the defect outline is zero (e.g., there is no defect in the image or the defect is a single pixel), the morphological complexity of the defect is directly assigned to zero, which is used to indicate that the defect is a point defect or a defect-free area, and its morphological complexity is the lowest.

[0047] Optionally, when a solder joint image includes multiple defect regions, the defect morphology complexity of each defect region can be calculated separately, and then the average defect morphology complexity of multiple defect regions can be determined as the defect morphology complexity of the solder joint image.

[0048] Optionally, the physical features of the image may also include penetration depth, which refers to the dimension of the weld joint fusion zone in the depth direction. For full penetration welds, the penetration depth is the vertical distance from the base material surface to the deepest point of the fusion line; for fillet welds, the penetration depth is the normal distance from the weld toe to the deepest point of the penetration depth. The penetration depth can be determined based on the width of the heat-affected zone and the grayscale gradient of the central region of the weld joint image.

[0049] It should be understood that the heat-affected zone (HAZ) is a transitional area near the weld that is heated but not melted. Its metallographic structure differs from that of the base metal and the fusion zone of the weld, which is reflected in the grayscale image as a different degree of dispersion in grayscale distribution compared to the surrounding area. The greater the heat input (higher current, slower welding speed), the wider the HAZ and the deeper the weld penetration.

[0050] Optionally, the geometric center of the weld fusion zone can be taken as the weld center; starting from the weld center, multiple rays are drawn out as scan lines, extending outward to the base material area. The method of drawing out the rays is, for example, drawing 8 or 16 rays at equal angular intervals within a 360° range, with the included angle between adjacent rays being 45° or 22.5°; for each scan line, the pixel distance of each pixel point on the line relative to the weld center and its corresponding gray value are recorded, forming a scan line corresponding to a gray value sequence and a pixel distance sequence.

[0051] For the grayscale value sequence on each scan line, the first derivative (grayscale gradient) is calculated using the central difference method. The position on each scan line where the first derivative first exceeds a preset inner boundary gradient threshold (e.g., 30% of the maximum grayscale gradient on that scan line) is defined as the inner boundary, and the pixel distance corresponding to the inner boundary is defined as the inner boundary pixel distance. The position on each scan line where the first derivative first falls below a preset outer boundary gradient threshold (e.g., 10% of the maximum grayscale gradient on that scan line) is defined as the outer boundary, and the pixel distance corresponding to the outer boundary is defined as the outer boundary pixel distance.

[0052] Based on the inner and outer boundary pixel distances, determine the radial width of each scan line; aggregate the radial widths of all scan lines (e.g., take the arithmetic mean or median value) to determine the pixel width of the heat-affected zone; convert the width of this pixel unit into the width of the physical distance through camera calibration parameters (pixel equivalent, in millimeters per pixel) to obtain the width of the heat-affected zone in millimeters.

[0053] Optionally, after determining the weld joint width, a circular region is delineated with the weld joint center as the origin and a radius equal to a preset proportion (e.g., 1 / 3 to 1 / 2) of the weld joint fusion zone radius. This circular region is designated as the central region. The gradient magnitude is calculated for each pixel within the central region using either the Sobel or Scharr operator. The arithmetic mean of the gradient magnitudes of all pixels within the central region is determined as the grayscale gradient of the central region. This grayscale gradient of the central region characterizes the intensity of the transition from bright to dark in the central region of the molten pool, indirectly reflecting the weld depth—the greater the weld depth, the more significant the thermal impact in the central region of the molten pool.

[0054] It should be understood that the penetration depth cannot be directly observed from the solder joint image. However, the width of the heat-affected zone and the grayscale gradient of the central region of the solder joint image can be input into the penetration depth prediction model to obtain the penetration depth of the solder joint image.

[0055] Optionally, the melt depth prediction model is a multiple linear regression fitting model. By selecting multiple representative real solder joint samples, metallographic sections are made, and the real melt depth is measured under a microscope. At the same time, the width of the heat-affected zone and the gray-scale gradient of the central region in the solder joint image corresponding to each real solder joint sample are extracted. The multiple linear regression method is used to establish the melt depth prediction model, and the least squares method is used to solve the regression coefficients. The regression determination coefficient is required to be greater than or equal to 0.85 to ensure the prediction accuracy.

[0056] Optionally, the melting depth prediction model satisfies the following formula:

[0057]

[0058] in, Indicates the depth of penetration. Indicates the width of the heat-affected zone. Indicates the gray-level gradient in the central region. , , is the regression coefficient.

[0059] Optionally, based on the welding environment parameters and physical characteristics of each weld point image, a method for establishing multiple sets of mapping relationships through regression analysis is as follows: using regression analysis methods, such as multiple linear regression, to establish the mapping relationship between weld width and welding current, welding voltage, and welding speed, as well as the mapping relationship between defect morphology complexity and welding current, welding voltage, and welding speed.

[0060] Optionally, for weld penetration depth, since it cannot be directly observed from weld joint images, the weld penetration depth prediction model can be directly used as its mapping rule; then these mapping relationships are stored in the welding process rule library.

[0061] In this embodiment, by explicitly quantifying the physical prior knowledge in the welding process into mapping rules, the physical authenticity assessment of the subsequent synthesized image has a clear and calculable standard reference, which can then help assess whether the synthesized image of the adversarial generative network conforms to physical laws.

[0062] S103. Determine the physical law deviation of the synthesized image of the generator adversarial network based on the welding process rule library.

[0063] The physical law deviation is used to characterize the degree to which the physical features of the synthesized image deviate from the physical laws. The larger the physical law deviation, the more serious the deviation of the synthesized image from the physical laws of welding.

[0064] It should be understood that a synthesized image refers to a simulated weld point image generated by the generator in a generative adversarial network based on the input latent space variables (i.e., welding environment parameters). By comparing the physical characteristics of the synthesized image with the physical characteristics that should theoretically be obtained from the welding environment parameters used when generating the image, the deviation of the physical laws of the synthesized image can be obtained.

[0065] In one alternative implementation, the actual physical features of the synthesized image from the generative adversarial network can be extracted; the welding process rule base can be queried based on the target welding environment parameters used when generating the synthesized image to obtain the predicted physical features; and the deviation of the physical law can be determined based on the deviation between the actual physical features and the predicted physical features.

[0066] Optionally, the method for extracting the image physical features of the synthesized image of the generative adversarial network is the same as the method for extracting the image physical features of the weld point image in S102 above. The extracted image physical features should at least include weld width and defect morphology complexity.

[0067] It should be understood that actual image physical features include the actual value of weld width and the actual value of defect morphological complexity, while predicted image physical features include the predicted value of weld width and the predicted value of defect morphological complexity.

[0068] Specifically, the target welding environment parameters input when generating the synthetic image are obtained, namely the target current, target voltage, and target welding speed. The welding process rule base is queried based on the target welding environment parameters. Based on the mapping relationship included in the welding process rule base, the predicted values ​​of weld width and defect morphology complexity corresponding to the target welding environment parameters are obtained.

[0069] Optionally, the predicted physical features of the image may also include the predicted value of the melt depth. The predicted value of the melt depth corresponding to the width of the heat-affected zone and the gray-level gradient of the central region is obtained by extracting the width of the heat-affected zone and the gray-level gradient of the central region of the synthetic image.

[0070] Then, the deviation between the actual value and the predicted value of each physical feature of the synthesized image is calculated separately, and all deviations are aggregated to obtain the physical law deviation.

[0071] It should be understood that the greater the deviation between the actual value and the predicted value of the physical features of the image, the more serious the deviation of the synthesized image from the physical laws of welding, and the greater the deviation from the physical laws should be.

[0072] Alternatively, a normalization method based on characteristic range can be used to determine the deviation of physical laws.

[0073] For example, deviations from physical laws satisfy the following formula:

[0074]

[0075] in, Represents a composite image Deviation from physical laws Indicates the number of physical features of an image. Represents a composite image The corresponding number Predicted values ​​of physical features of an image Represents a composite image The corresponding number The actual value of each physical feature of the image. Indicates the first The statistical maximum value of each image physical feature in the training dataset. Indicates the first The statistical minimum of each image physical feature in the training dataset. It is a very small constant, with an example value of 10. -8 This is used to prevent the denominator from being zero.

[0076] In this formula, Indicates the first The deviation between the predicted and actual values ​​of a physical feature of an image. Indicates the characteristic range, used to... The value range is limited to between 0 and 1. Taking the absolute value indicates that only the degree of deviation is considered, not the direction of deviation. Finally, the deviation degree of all physical features of the images is averaged to obtain the deviation of the physical law.

[0077] Understandably, by comparing the actual physical features of the synthesized image with the physical features predicted by the welding process rule base, the generator can perceive the level of physical authenticity of its output synthesized image during the training process, thereby providing a clear physical constraint direction for subsequent parameter updates.

[0078] S104. Train the generative adversarial network based on the physical law deviation and the adversarial loss of the generative adversarial network to obtain the trained sample generation model.

[0079] It should be understood that a generative adversarial network includes a generator and a discriminator. The adversarial loss is used to characterize the difference between the discriminator's judgment of the authenticity of the synthesized image and the expected result. The sample generation model refers to the generator used to generate samples of solder joint quality defects after training.

[0080] Specifically, in each training iteration, the generator receives the target welding environment parameters and target defect type from the latent space input, generates a synthetic image, and the discriminator distinguishes between the real sample and the synthetic image to obtain the adversarial loss. At the same time, the physical law deviation of the synthetic image is determined, and then the physical law deviation and the adversarial loss are used together as training signals to guide the parameter updates of the generator and discriminator. The training is iterated until the convergence condition is met, and the converged generator is used as the sample generation model.

[0081] Optionally, the generative adversarial network can adopt the Wasserstein generative adversarial network (WGAN-GP) framework, and the adversarial loss can be determined based on the Wasserstein generative adversarial network (WGAN-GP) framework.

[0082] Optionally, the convergence condition can be: the total loss of the generative adversarial network changes less than a stable threshold in multiple consecutive training rounds (e.g., the relative change in the total loss value is less than 0.1% in 10 consecutive training rounds), and the generated sample set of solder joint quality defects passes the quality inspection, while the latent space decoupling score meets the judgment condition.

[0083] Understandably, physical law deviation is used to constrain the synthesized image to conform to the physical laws of welding, while adversarial loss is used to constrain the synthesized image to be visually close to the real weld point image. The synergistic optimization of the two enables the generator to gradually learn to generate weld point defect images that are both visually realistic and physically credible.

[0084] S105. Generate a sample set of weld joint quality defects based on the sample generation model.

[0085] Specifically, the target welding environment parameters and target defect types are input into the sample generation model. Based on the input conditions, the sample generation model selects the corresponding environmental factor representation and defect content representation in the latent space, and generates multiple weld point defect images that conform to the target working conditions through decoding, thus forming a weld point quality defect sample set.

[0086] In one alternative implementation, after generating the solder joint quality defect sample set, a quality inspection can be performed on the solder joint quality defect sample set: when the solder joint quality defect sample set passes the quality inspection, the solder joint quality defect sample set is determined as a usable sample set.

[0087] Specifically, the quality inspection method is as follows: determine the distribution difference index based on the sample set of solder joint quality defects and the real sample set; determine the defect coverage rate based on the number of defect types covered in the sample set of solder joint quality defects; when the distribution difference index meets the preset distribution conditions and the defect coverage rate meets the preset coverage conditions, the sample set of solder joint quality defects is confirmed to have passed the quality inspection.

[0088] It should be understood that the distribution difference index is used to characterize the visual distribution difference between the solder joint quality defect sample set and the real sample set. The higher the distribution difference index, the more similar the solder joint quality defect sample set is to the real solder joint image in terms of overall visual style, texture features, and defect morphology distribution.

[0089] Alternatively, the Fréchet Inception Distance (FID) can be used as an indicator of distribution dissimilarity. The Fréchet Inception Distance measures the difference between the generated sample distribution and the real sample distribution; the smaller the distance, the closer the distributions are.

[0090] Alternatively, a publicly available FID calculation tool can be used to input a sample set of weld quality defects and a real sample set to obtain the Fraser starting distance.

[0091] Optionally, the preset distribution condition can be set to a Fraser starting distance of less than or equal to 30.

[0092] It should be understood that defect coverage is used to assess whether the defect types in the solder joint quality defect sample set are sufficient. The solder joint quality defect sample set needs to cover the target defect type (i.e. the defect type expected to be generated) in order to effectively supplement various defect data and solve the problem of sample imbalance.

[0093] Optionally, the defect coverage rate can be determined by the ratio of the number of defect types in the solder joint quality defect sample set to the total number of target defect types.

[0094] Optionally, the preset coverage condition can be a defect coverage rate greater than 75%.

[0095] It is understandable that when the distribution difference index meets the preset distribution conditions and the defect coverage meets the preset coverage conditions, it means that the visual realism of the solder joint quality defect sample set meets the requirements and the coverage of the target defect type also meets the requirements. At this time, the solder joint quality defect sample set can be determined as a usable sample set.

[0096] When the distribution difference index does not meet the preset distribution conditions or the defect coverage does not meet the preset coverage conditions, it indicates that the visual realism of the solder joint quality defect sample set cannot meet the requirements, or the coverage of the target defect type cannot meet the requirements. At this time, it can be determined that the solder joint quality defect sample set has not passed the quality inspection. In this case, the solder joint quality defect sample set can be discarded to avoid low-quality generated samples from polluting the downstream training data.

[0097] This quality check allows us to select high-quality generated samples with realistic visual distribution and high defect coverage. Then, we can mix the available sample set with the original sample set to obtain an augmented dataset. Finally, we can use the augmented dataset to train the defect detection model, thereby improving the generalization performance and robustness of the defect detection model.

[0098] The methods provided in S101-S105 above acquire weld point images and corresponding welding environment parameters, and establish a welding process rule base. This quantifies the overall mapping law between welding environment parameters and the physical features (geometric shape and defect distribution attributes) of weld point images, providing a queryable and calculable evaluation standard for the physical authenticity of synthesized images. During training, the physical law deviation of synthesized images is determined in real time according to the welding process rule base, enabling the generator to perceive in real time whether its output violates the physical laws of welding. The physical law deviation and the adversarial loss of the generative adversarial network are used as training objectives, ensuring that the generator, while improving visual authenticity, must meet the constraints of physical laws, ensuring that the synthesized images still have process credibility under multi-factor coupling conditions. Finally, the weld point quality defect sample set generated by the trained sample generation model not only has the high diversity brought by the generative adversarial network, but its geometric features also strictly correspond to the input welding environment parameters, resulting in high quality stability. It can be directly used to supplement the training data of rare defects (such as cracks and lack of fusion) in the long-tail distribution, thereby improving the training effect and generalization of the defect detection model.

[0099] Combination Figure 1 ,like Figure 2 As shown, the Generative Adversarial Network (GAN) is trained based on the physical law deviation and the adversarial loss of the GAN to obtain the trained sample generation model, which can be specifically implemented through S201-S203.

[0100] S201. In each training iteration, obtain the synthesized image of the generative adversarial network and the adversarial loss of the generative adversarial network in the current iteration.

[0101] Specifically, during the training iteration of the generative adversarial network, the synthesized image generated in the current iteration is obtained from the generator, while the adversarial loss is obtained from the discriminator.

[0102] S202. Determine the total loss based on the physical law deviation and adversarial loss of the synthesized image.

[0103] Optionally, the total loss can be determined by summing the deviation from physical laws and the countermeasure loss.

[0104] It should be understood that while deviations in physical laws can directly constrain the degree to which physical features of the synthesized image, such as weld width and defect morphology, conform to welding process rules, their evaluation perspective is unidirectional, i.e., the physical features of the image are deduced from welding environment parameters. However, the coupled effects of multiple factors such as current, voltage, and welding speed during the welding process produce complex overall visual effects, such as texture distribution, shadow levels, and heat-affected zone transition patterns. These visual features are difficult to fully capture through a single deviation in physical laws.

[0105] In one implementation of this application, mapping error can be introduced as a cross-validation mechanism. Mapping error characterizes the degree of consistency between the synthesized image and the target welding environment parameters in terms of overall visual performance. Welding environment parameters are inferred from the weld point image, and the consistency between the inferred result and the input is compared to capture the overall visual consistency deviation after multi-factor coupling. Mapping error and physical law deviation complement each other: mapping error constrains the soft consistency of the synthesized image with the target working condition in terms of overall visual performance, while physical law deviation constrains the hard compliance of the image's physical features. The two work together to form a coupled constraint system.

[0106] Specifically, the above-mentioned determination of the total loss based on the physical law deviation and adversarial loss of the synthesized image may include: inputting the synthesized image into a parameter prediction network to obtain predicted values ​​of welding environment parameters; determining the mapping error based on the deviation between the predicted values ​​of welding environment parameters and the target welding environment parameters; and determining the total loss based on the physical law deviation, mapping error, and adversarial loss of the synthesized image.

[0107] It should be noted that the parameter prediction network refers to a pre-trained neural network used to infer welding environment parameters from weld joint images. This parameter prediction network is obtained by pre-training on a real dataset, with real weld joint images as input and predicted values ​​of welding environment parameters (welding current, voltage, welding speed) as output.

[0108] For example, the parameter prediction network can adopt the ResNet18 architecture, perform transfer learning on the basis of ImageNet pre-trained weights, modify the output layer to have 3 neurons (corresponding to current, voltage, and welding speed), and train until convergence with mean squared error as the loss function.

[0109] Specifically, the synthesized image is input into the parameter prediction network. The parameter prediction network extracts the boundary features and gray-scale distribution features of the weld fusion zone in the synthesized image through its internal convolutional layers, and determines the predicted values ​​of welding environment parameters based on the boundary features and gray-scale distribution features.

[0110] It should be understood that the predicted values ​​of welding environment parameters include the predicted values ​​of each welding environment parameter, and the target welding environment parameters include the target values ​​of each welding environment parameter.

[0111] It should be noted that the predicted value of the welding environment parameter is also a normalized value to ensure consistency with the parameters used in the generative adversarial network.

[0112] Optionally, the absolute value of the difference between the predicted value and the target value of each welding environment parameter can be determined first, and then the absolute value of the difference corresponding to all welding environment parameters can be averaged to obtain the mapping error.

[0113] Optionally, after determining the mapping error, a weighted average can be taken of the physical law deviation, mapping error, and adversarial loss to obtain the total loss. The total loss obtained in this way comprehensively considers the consistency of physical laws of the synthesized image, the consistency of environmental parameter back-inference, and the realism of visual distribution.

[0114] For example, the weight of physical law deviation can be set to 0.3, the weight of mapping error can be set to 0.5, and the weight of countermeasure loss can be set to 1.

[0115] In another implementation of this application, to further improve the directional controllability of the synthesized image, decoupling constraints can be introduced when determining the total loss. Specifically, determining the total loss based on the physical law deviation, mapping error, and adversarial loss of the synthesized image can include: splitting the latent space of the generative adversarial network into an environmental factor latent space and a defect content latent space; transforming the input variables of the environmental factor latent space and the defect content latent space into environmental feature vectors and defect feature vectors; determining the mutual information between each dimension within the environmental feature vector, between each dimension within the defect feature vector, and between the environmental feature vector and the defect feature vector; determining the average value of all mutual information as a regularization term; and determining the total loss based on the physical law deviation, mapping error, regularization term, and adversarial loss of the synthesized image.

[0116] It should be understood that since the input variables in the latent space are sampled independently (and are independent of the network parameters in the generative adversarial network), the partial derivatives of mutual information with respect to the network parameters are always zero, and gradients cannot be generated. Therefore, the latent variables need to be transformed into environmental feature vectors and defect feature vectors through fully connected layers before the mutual information between dimensions is determined, and then the generative adversarial network is trained to achieve decoupling.

[0117] Specifically, the latent space of environmental factors and the latent space of defect content correspond to different dimensional segments of the generator input vector; the generator receives the input variables corresponding to the latent space of environmental factors and the latent space of defect content, and transforms the input variables into environmental feature vectors and defect feature vectors respectively through the independent mapping structure at its front end.

[0118] It should be understood that when constructing a generative adversarial network, the first step is to define the latent space of the generator's input. The dimensions of the latent space should include the dimensions of all environmental feature vectors and the dimensions of defect feature vectors.

[0119] For example, the environmental factor latent space may include three dimensions, each corresponding to a welding environment parameter. These three dimensions are used to control current, control voltage, and control welding speed, respectively. The defect content latent space may include five dimensions, each corresponding to a defect type. These five dimensions are used to control porosity, control cracks, control incomplete penetration, control lack of fusion, and control slag inclusions, respectively.

[0120] To achieve latent space splitting, two independent input branches can be set in the generator's network structure: the first branch connects to the environmental factor latent space, consisting of two fully connected layers, used to encode environmental factors into intermediate feature vectors; the second branch connects to the defect content latent space, also consisting of two fully connected layers, used to encode defect types into intermediate feature vectors. The outputs of the two branches are concatenated along the channel dimension and then fed into subsequent convolutional layers for image generation.

[0121] However, simple structural decomposition cannot guarantee true statistical independence of each dimension during training. In actual training, due to the co-occurrence deviation between current and welding speed in real data (e.g., high current is often accompanied by high welding speed), statistical entanglement may still occur between dimensions within the latent space of environmental factors, as well as between environmental factors and defect content. This can lead to unintentional linkage between welding speed and current changes, or passive drift of welding environment parameters when the defect type changes. This dimensional entanglement can destroy the purity of targeted generation, making it impossible to independently generate rare defect samples under specific working conditions as needed. Therefore, to ensure that the two subspaces after decomposition are truly decoupled functionally, mutual information regularization constraints need to be introduced during training.

[0122] It should be understood that mutual information is used to measure the degree of statistical dependence between two random variables.

[0123] Optionally, random sampling can be performed on each dimension of the latent space, which is then mapped to environmental feature vectors and defect feature vectors via a generative adversarial network. Then, the probability distribution of the continuous output values ​​of each dimension of the environmental feature vector and the defect feature vector can be estimated using a differentiable method.

[0124] It should be noted that the values ​​of each dimension of the defect content latent space are continuous floating-point values ​​sampled from a continuous probability distribution, used to characterize the generation intensity or occurrence probability of the corresponding defect type; the dimensions of the defect feature vector after being mapped by the generator are also continuous real number outputs, which can be directly used for differentiable probability density estimation and mutual information calculation.

[0125] For example, a soft binning method based on Gaussian kernels can be used. This involves calculating the Gaussian membership degree of each sampling point to a preset bin center and constructing a differentiable soft histogram using a weighted summation method, thus avoiding gradient vanishing caused by hard truncation. Alternatively, a kernel density estimation (KDE) method can be used to smooth the continuous probability density with a Gaussian kernel function, ensuring that the probability estimation process is differentiable with respect to network parameters. Based on the estimated probability distribution, the joint probability and marginal probability are calculated, and then the mutual information is calculated.

[0126] It is understandable that the Gaussian membership function in the soft binning method or the kernel function in the kernel density estimation are both continuously differentiable functions of the network parameters. The partial derivatives of the output probability distribution with respect to the network mapping parameters are non-zero, thereby ensuring that the mutual information regularization term can generate an effective gradient through backpropagation to guide the update of network parameters.

[0127] Optionally, for any two dimensions of the environmental feature vector and the defect feature vector, the mutual information satisfies the following formula:

[0128]

[0129] in, Representing dimensions With dimension Mutual information between them Representing dimensions The specific values, Representing dimensions The specific values, Representing dimensions Values And dimensions Values The probability of that, i.e., the joint probability; Indicates no consideration Time, dimension Values The probability of that, i.e., the marginal probability; Indicates no consideration Time, dimension Values The probability of; This represents the logarithmic operation with the natural constant e as the base. Represents the dimension All possible values ​​and dimensions Iterate through and sum all possible values.

[0130] In this formula, This represents the ratio of the product of the joint probability and the marginal probability. If the dimension... With dimension If they are statistically independent, then the joint probability equals the product of the marginal probabilities, i.e. If dimension With dimension If statistical dependence (entanglement) exists, the joint probability deviates from the product of marginal probabilities. Greater than 1 or less than 1. The logarithmic operation maps the probability ratio to an information gain scale; when the dimensions are independent, the ratio is always equal to 1. , The logarithmic terms do not contribute to mutual information; when the dimensions are positively correlated and coupled (deviation in the same direction), A value greater than 1 indicates a positive logarithmic term; when the dimensions are negatively correlated and coupled (inversely deviating), If the logarithm is less than 1, the logarithmic term is negative; the sign of the logarithmic term directly reflects whether the two dimensions are "entangled in the same direction" or "entangled in opposite directions" under this combination of values.

[0131] Then, using the joint probability as the weight, the logarithmic terms are weighted so that combinations of values ​​with high probabilities contribute more to the mutual information, while combinations with low probabilities contribute less. Finally, all possible combinations of values ​​for the two dimensions of the latent space are traversed, and the weighted values ​​for each combination are summed to obtain the dimension. With dimension The amount of mutual information shared between them, measured in bits or nits.

[0132] It should be noted that in the actual calculation of the above formula, if the marginal probability of a certain dimension is zero, the corresponding joint probability is also zero. In this case, probability smoothing (such as Laplace smoothing) can be used to avoid the case where the logarithm is zero and the denominator is zero, thus ensuring the stability of mutual information calculation.

[0133] It should be noted that the environmental feature vector and the defect feature vector are explicit outputs of the internal mapping of the generative adversarial network. The mutual information between their dimensions can generate non-zero gradients for the network mapping parameters through backpropagation. At the same time, by discretizing the continuous output values, the definition of the mutual information probability mass function, which is originally applicable to discrete random variables, can be applied to continuous floating-point outputs, avoiding the probability degradation problem caused by direct sampling.

[0134] After obtaining the mutual information between all dimension pairs (i.e., the mutual information between each dimension within the environmental feature vector, between each dimension within the defect feature vector, and between the environmental feature vector and the defect feature vector), the arithmetic mean of the mutual information between all dimension pairs is calculated to obtain the average mutual information value, and this average mutual information value is used as the regularization term.

[0135] It should be understood that the regularization term is used to penalize high statistical dependencies between latent space dimensions, i.e., high entanglement.

[0136] Finally, the physical law deviation, mapping error, regularization term, and adversarial loss of the synthesized image are weighted and summed to obtain the total loss.

[0137] It should be understood that, due to the differences in the dimensions and orders of magnitude of the various loss terms, direct addition may lead to one term dominating the optimization process. Therefore, numerical balancing and dimensional coordination can be achieved through weighting coefficients to ensure that each term contributes equally to the total loss.

[0138] It should be noted that each weight coefficient can be adjusted independently according to the degree of influence of each constraint on the quality of the synthesized image. For example, the weight of physical law deviation can be set to 0.3, the weight of mapping error can be set to 0.5, the weight of regularization term can be set to 0.2, and the weight of adversarial loss can be set to 1.

[0139] Meanwhile, the weights of the regularization term are functionally equivalent to the conversion ratio that scales the mutual information metric from bit units to a dimensionless loss contribution.

[0140] The method described above for determining the total loss incorporates physical law deviation, mapping error, and mutual information mean into the total loss, requiring the generator to satisfy multiple constraints during adversarial training: physical law deviation ensures that hard physical quantities such as weld width and defect morphology conform to process rules; mapping error verifies the overall visual consistency after multi-factor coupling by inferring from the image; and mutual information regularization ensures that environmental parameters and defect types are independently controllable. The synergistic effect of these three factors makes the synthesized image both physically realistic and directionally controllable.

[0141] In one optional implementation, during training, a decoupling evaluation can be triggered every preset number of training epochs (e.g., every 1000 iterations or every 5 training epochs). Based on the decoupling evaluation results, the regularization weights are dynamically optimized. Specifically, the values ​​of other latent space dimensions are fixed, the value range of the target dimension is traversed, and multiple sets of images are generated; the factor values ​​corresponding to the target dimension in the multiple sets of images are measured; the correlation between the value sequence of the target dimension and the factor value sequence is determined; the decoupling score is determined based on the mean absolute value of the correlation of all latent space dimensions; and the weight of the regularization term is adjusted based on the decoupling score until the decoupling score meets the judgment condition.

[0142] Here, the target dimension is any one of all latent space dimensions, and the other latent space dimensions are all latent space dimensions other than the target dimension.

[0143] It should be understood that during periodic decoupling evaluation, it is necessary to pause the regular forward-backward propagation training process and then determine the relevance of each dimension.

[0144] Taking the target dimension as an example, firstly, the values ​​of other latent space dimensions should be kept fixed, and only the value of the target dimension should be changed. Sampling should be performed within the range of the target dimension's values. For each set of values, the corresponding image should be generated by the generator in the current training state, thereby obtaining multiple sets of images.

[0145] Subsequently, for multiple sets of images, the factor values ​​corresponding to the target dimension are measured to obtain the value sequence of the target dimension and the factor value sequence. For example, if the target dimension is the dimension of current in the latent space of environmental factors, then the physical quantities reflecting the current level in each set of images, such as melt width, are measured; if the target dimension is the dimension of crack in the latent space of defect content, then the defect type in each set of images is measured.

[0146] Subsequently, the correlation between the target dimension value sequence and the corresponding factor value sequence is determined.

[0147] It should be understood that since each dimension of the environmental factor latent space corresponds to continuous process parameters such as welding current, voltage, and welding speed, their values ​​change continuously in the real number domain, and there is an approximately linear physical mapping relationship between them and the physical features of the output image (such as weld width). Therefore, if a target dimension belongs to the environmental factor latent space, the Pearson correlation coefficient can be used to measure the degree of correlation. The closer the absolute value of the Pearson correlation coefficient is to 1, the stronger the linear correlation between the target dimension and the corresponding factor, that is, the higher the semantic purity of the target dimension.

[0148] It should be noted that although each dimension of the defect content latent space semantically corresponds to a different defect type, its values ​​are all continuous floating-point values ​​sampled from a continuous probability distribution, used to characterize the generation intensity of the corresponding defect type. Therefore, when traversing the value range of the target dimension, images with different defect presentation intensities can be generated by changing the continuous floating-point value of that dimension (e.g., uniform sampling within a preset interval). Since each dimension of the defect content latent space corresponds to defect types such as porosity, cracks, and lack of fusion, they are nominal output variables without a progressive relationship of magnitude and cannot establish a linear correlation with continuous input values. This does not satisfy the prerequisites of "continuous values" and "linear relationship" required by the Pearson correlation coefficient. Therefore, if the target dimension belongs to the defect content latent space, the classification accuracy is used to measure the degree of correlation, that is, the proportion of images correctly identified as the target defect type by the pre-trained defect classifier among the generated multiple sets of images.

[0149] After obtaining the correlation degree of all latent space dimensions, the average of the absolute values ​​of all correlation degrees is calculated, and this average value is determined as the decoupling score.

[0150] It should be understood that the decoupling score is used to comprehensively evaluate the overall decoupling effect of the latent space. The higher the decoupling score, the stronger the semantic independence of each dimension of the latent space.

[0151] Optionally, a judgment condition can be set. For example, the judgment condition can be set to a decoupling score greater than or equal to 0.8. If the decoupling score does not meet the judgment condition, it indicates that there is still statistical entanglement between the dimensions of the latent space. In this case, the weight coefficient of the regularization term in the total loss can be increased by a preset step size (e.g., 0.1) to enhance the purity of the association between the dimensions and the factors. After the adjustment is completed, the normal training iteration is resumed, and the updated weights are used to continue optimization in subsequent training.

[0152] If the decoupling score meets the judgment condition, it means that the latent space has been fully decoupled. At this time, keep the weight of the current regularization term unchanged and continue to perform regular training iterations until the whole converges.

[0153] Understandably, by periodically embedding decoupling evaluation into the training process and using the decoupling score as a feedback signal to dynamically adjust the weight of the regularization term, it avoids both the suppression of adversarial training due to excessively strong regularization terms and the dimensional entanglement caused by insufficient decoupling strength, thus ensuring that each dimension of the latent space gradually moves towards independent semantic control throughout the training process.

[0154] S203. Update the network parameters of the generated adversarial network based on the total loss.

[0155] Specifically, gradient backpropagation and updates are performed on the network parameters of the generator and discriminator based on the total loss, so that the generator can generate images that meet the requirements in the next iteration.

[0156] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0157] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for enhancing solder joint quality defect sample data based on generative adversarial networks, characterized in that, include: Acquire multiple solder joint images and the corresponding welding environment parameters for each solder joint image; A welding process rule library is established based on multiple weld point images and the welding environment parameters corresponding to each weld point image. The welding process rule library includes multiple sets of mapping relationships between welding environment parameters and image physical features. The image physical features are used to characterize the geometric shape and defect distribution attributes in the weld point images. The physical law deviations of the synthesized images generated by the generative adversarial network are determined based on the welding process rule base. The generative adversarial network is trained based on the deviation of physical laws and the adversarial loss of the generative adversarial network to obtain the trained sample generation model; Based on the sample generation model, a sample set of weld joint quality defects is generated.

2. The method for enhancing solder joint quality defect sample data based on generative adversarial networks according to claim 1, characterized in that, The step of establishing a welding process rule library based on multiple weld point images and the corresponding welding environment parameters for each weld point image includes: Determine the image physical features corresponding to each weld point image, wherein the image physical features include at least weld width and defect morphology complexity; Based on the welding environment parameters and physical characteristics of each weld point image, multiple sets of mapping relationships are established through regression analysis.

3. The method for enhancing solder joint quality defect sample data based on generative adversarial networks according to claim 2, characterized in that, The solder joint image also includes the perimeter and area of ​​the defect region. Determining the physical features of each solder joint image includes: Edge detection is performed on the images of each weld point, and the weld width is determined based on the edge detection results and camera calibration parameters; The morphological complexity of the defect is determined based on the perimeter and area of ​​the defect region.

4. The method for enhancing solder joint quality defect sample data based on generative adversarial networks according to claim 1, characterized in that, The step of determining the physical law deviation of the synthesized image according to the welding process rule library includes: Extract the actual physical features of the synthesized image; The physical features of the predicted image are obtained by querying the welding process rule base based on the target welding environment parameters used when generating the synthetic image. The deviation of physical laws is determined based on the deviation between the actual physical features of the image and the predicted physical features of the image.

5. The method for enhancing solder joint quality defect sample data based on generative adversarial networks according to claim 4, characterized in that, The process of training the generative adversarial network (GAN) based on physical law deviations and the adversarial loss of the GAN to obtain the trained sample generation model includes: In each training iteration, the synthesized image of the generative adversarial network and the adversarial loss of the generative adversarial network in the current iteration are obtained; The total loss is determined based on the physical law deviation and adversarial loss of the synthesized image; The network parameters of the generative adversarial network are updated based on the total loss.

6. The method for enhancing solder joint quality defect sample data based on generative adversarial networks according to claim 5, characterized in that, The determination of the total loss based on the physical law deviation and adversarial loss of the synthesized image includes: The synthesized image is input into the parameter prediction network to obtain predicted values ​​of welding environment parameters; The mapping error is determined based on the deviation between the predicted welding environment parameters and the target welding environment parameters; The total loss is determined based on the physical law deviation, mapping error, and adversarial loss of the synthesized image.

7. The method for enhancing solder joint quality defect sample data based on generative adversarial networks according to claim 6, characterized in that, The determination of the total loss based on the physical law deviation, mapping error, and adversarial loss of the synthesized image includes: The latent space of generative adversarial networks is divided into an environmental factor latent space and a defect content latent space; The input variables of the latent space of environmental factors and the latent space of defect content are transformed into environmental feature vectors and defect feature vectors. Determine the mutual information between the dimensions within the environmental feature vector, between the dimensions within the defect feature vector, and between the environmental feature vector and the defect feature vector; The average value of all mutual information is used as the regularization term; The total loss is obtained by weighted summing of the physical law deviation, mapping error, regularization term, and adversarial loss of the synthesized image.

8. The method for enhancing solder joint quality defect sample data based on generative adversarial networks according to claim 7, characterized in that, The method further includes: The values ​​of other latent space dimensions are fixed, and the range of values ​​of the target dimension is traversed to generate multiple sets of images. The target dimension is any one of all latent space dimensions, and the other latent space dimensions are dimensions other than the target dimension. Measure the factor values ​​corresponding to the target dimension in the multiple sets of images; Determine the degree of correlation between the target dimension's value sequence and the factor value sequence; The decoupling score is determined based on the mean absolute value of the correlation of all latent space dimensions; The weight of the regularization term is adjusted based on the decoupling score until the decoupling score meets the determination criteria.

9. The method for enhancing solder joint quality defect sample data based on generative adversarial networks according to claim 1, characterized in that, After obtaining the sample set of weld joint quality defects, the method further includes: Quality inspection was performed on the sample set of weld joint quality defects: When the sample set of solder joint quality defects passes the quality inspection, the sample set of solder joint quality defects is determined as a usable sample set.

10. The method for enhancing solder joint quality defect sample data based on generative adversarial networks according to claim 9, characterized in that, The quality inspection of the sample set of weld joint quality defects includes: Determine the distribution difference index based on the sample set of weld joint quality defects and the real sample set; The defect coverage rate is determined based on the number of defect types covered in the sample set of weld joint quality defects. When the distribution difference index meets the preset distribution conditions and the defect coverage rate meets the preset coverage conditions, the weld joint quality defect sample set is confirmed to have passed the quality inspection.

Images