Welding quality prediction method, model training method, device and chip

Abstract

The invention belongs to the field of laser welding, and discloses a welding quality prediction method, a model training method, a device and a chip, and the method comprises the steps: based on a post-welding image, extracting welding spots, and forming a welding spot diagram; based on the welding spot diagram, determining neural network features and image features corresponding to the post-welding image; and inputting the neural network features and the image features into a welding quality prediction model to obtain a welding quality prediction value corresponding to the post-welding image. According to the method, the welding quality prediction value of the post-welding material corresponding to the post-welding image is predicted by analyzing the dual characteristics through the welding quality prediction model, the accuracy of the welding quality prediction value is improved, the accuracy of welding quality detection is further improved, the welding quality prediction of all the post-welding materials can be realized, and the accuracy of welding quality detection is improved. And physical damage to welded materials is not needed, extra detection equipment and material consumption are not needed, the welding quality detection cost is saved, manual intervention and labor cost are reduced, and the welding quality determination efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of laser welding, and more specifically, to a method for predicting welding quality, a method for training a model, an apparatus, and a chip. Background Technology

[0002] In modern industrial production, welding quality inspection is a crucial step in ensuring product quality and safety. Traditional welding quality inspection methods mainly rely on destructive testing, such as pull-out force testing. These methods require physically destroying the welded material to measure its strength and reliability. However, this approach is not only time-consuming and labor-intensive but also costly, and can only test a small number of samples, failing to achieve comprehensive quality monitoring.

[0003] Therefore, how to improve the efficiency of welding quality inspection and reduce inspection costs is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] To address the problems of low efficiency and high cost in welding quality inspection in existing technologies, this invention provides a method for welding quality prediction, a model training method, an apparatus, and a chip.

[0005] A method for predicting welding quality includes: Based on the post-weld image, the solder joints are extracted to form a solder joint image; Based on the solder joint diagram, determine the neural network features and image features corresponding to the post-soldering image; The neural network features and the image features are input into the welding quality prediction model to obtain the welding quality prediction value corresponding to the post-weld image.

[0006] Optionally, determining the neural network features corresponding to the post-weld image based on the solder joint image includes: Based on the feature extraction model, the solder joint image is subjected to dimensionality reduction and feature extraction processing to obtain the neural network features corresponding to the solder joint image. Based on the neural network features corresponding to the solder joint image, the neural network features corresponding to the post-soldering image are determined.

[0007] Optionally, the training method for the feature extraction model includes: Based on historical post-weld images, historical solder joints are extracted to form a historical solder joint map; The historical solder joint image is projected onto the feature vector space to obtain feature values; The eigenvalues ​​in the eigenvector space are restored to obtain the reconstructed image; The difference between the reconstructed image and the historical solder joint image is determined, and the feature extraction model is trained based on the difference input model to form the feature extraction model.

[0008] Optionally, based on a feature extraction model, the solder joint image is subjected to dimensionality reduction and feature extraction processing to obtain the neural network features corresponding to the solder joint image, including: The solder joint image is projected onto the feature vector space of the feature extraction model to obtain the feature values ​​of the solder joint image; The feature values ​​of the solder joint image are determined as the neural network features corresponding to the solder joint image.

[0009] Optionally, the post-weld image has multiple solder joints, and determining the image features corresponding to the post-weld image based on the solder joint image includes: Based on the solder joint diagram, determine the total number of solder joints and the width and height of each solder joint; The diameter of a solder joint is determined based on the total number of solder joints and the width and height of each solder joint. The diameter of the weld point is determined as the image feature corresponding to the post-weld image.

[0010] Optionally, determining the image features corresponding to the post-weld image based on the solder joint diagram includes: The grayscale value of each pixel in the solder joint image is fitted using a preset probability density function to obtain the scale parameter and shape parameter of the preset probability density function. The mode of the grayscale distribution of the solder joint image is determined based on the scale parameter of the preset probability density function and the shape parameter. The image features corresponding to the post-weld image are determined based on the mode of the grayscale distribution of the solder joint image.

[0011] Optionally, the method further includes: The grayscale standard deviation of the solder joint image is determined based on the scale parameter of the preset probability density function and the shape parameter. The image features corresponding to the post-weld image are determined based on the grayscale standard deviation of the solder joint image.

[0012] Optionally, determining the image features corresponding to the post-weld image based on the solder joint diagram includes: The gray-level co-occurrence matrix of the solder joint image is determined based on the gray-level co-occurrence matrix algorithm. The texture complexity of the solder joint image is determined based on the gray-level co-occurrence matrix. The image features corresponding to the post-weld image are determined based on the texture complexity of the solder joint image.

[0013] Optionally, the predicted welding quality value includes the predicted pull-out force value and the confidence level of the predicted pull-out force value, and the predicted welding quality model includes a Gaussian process regression model; The step of inputting the neural network features and the image features into the welding quality prediction model to obtain the welding quality prediction value corresponding to the post-weld image includes: The neural network features and the image features are standardized to obtain the input values; The mean and variance of the input values ​​are calculated using the kernel function of the Gaussian process regression model. The mean is determined as the predicted welding quality value corresponding to the post-weld image, and the confidence level of the predicted welding quality value is determined based on the variance.

[0014] An apparatus for predicting welding quality includes: The solder joint extraction module is used to extract solder joints from the post-weld image and generate a solder joint map. The feature module is used to determine the neural network features and image features corresponding to the post-weld image based on the solder joint diagram; The prediction module is used to input the neural network features and the image features into the welding quality prediction model to obtain the welding quality prediction value corresponding to the post-weld image.

[0015] A training method for a welding quality prediction model includes: Based on the post-weld image, the solder joints are extracted to form a training solder joint image; Based on the trained solder joint diagram, neural network features and image features are determined; Based on the pull-out force test value corresponding to the training solder joint image, and combined with the neural network features and image features of the training solder joint image, a training sample is formed; A training sample set is constructed using the training samples corresponding to each of the training solder joint diagrams. The initial welding quality prediction model is trained using the training sample set, and the initial welding quality prediction model after the iteration ends is used as the welding quality prediction model.

[0016] Optionally, the initial welding quality prediction model is a Gaussian process regression model; The step of training the initial welding quality prediction model using the training sample set includes: Standardize all training samples in the training sample set; Determine the kernel function of the Gaussian process regression model; Based on the standardized training samples, the hyperparameters of the kernel function are optimized by maximizing the marginal likelihood function to form the welding quality prediction model.

[0017] Optionally, determining the kernel function of the Gaussian process regression model includes: Determine if user input has been received, and select the kernel function type. If so, then the kernel function of the Gaussian process regression model is determined based on the kernel function selection type. If not, the kernel function of the Gaussian process regression model is determined based on the preset kernel function type.

[0018] A computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the welding quality prediction method described in any one of the preceding claims or the training method for the welding quality prediction model described in any one of the preceding claims.

[0019] A chip, comprising: A processor and a memory, the memory being used to store at least one instruction, which, when loaded and executed by the processor, implements a welding quality prediction method as described in any of the preceding claims or a training method for a welding quality prediction model as described in any of the preceding claims.

[0020] The welding quality prediction method provided in this invention extracts weld points from post-weld images to form a weld point map; based on the weld point map, it determines the neural network features and image features corresponding to the post-weld images; and it inputs the neural network features and image features into a welding quality prediction model to obtain the predicted welding quality value corresponding to the post-weld images. This invention analyzes both the neural network features and image features of the post-weld images using a welding quality prediction model to predict the welding quality of the corresponding post-weld materials, thereby effectively improving the accuracy of the predicted welding quality values ​​and thus enhancing the accuracy of welding quality inspection. Furthermore, this invention, based on image analysis and model prediction, can predict the welding quality of all post-weld materials without physical damage, additional inspection equipment, or material consumption, saving welding quality inspection costs, reducing manual intervention, lowering labor costs, and improving the efficiency of welding quality determination. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a method for predicting welding quality provided in an embodiment of the present invention; Figure 2 for Figure 1 A flowchart illustrating the first actual manifestation of S02 in a method for predicting welding quality; Figure 3 for Figure 1 A flowchart illustrating the second actual manifestation of S02 in a method for predicting welding quality; Figure 4 for Figure 1 A flowchart illustrating the third actual manifestation of S02 in a method for predicting welding quality; Figure 5 for Figure 1 A flowchart illustrating one actual manifestation of S03 in a method for predicting welding quality; Figure 6 A flowchart illustrating a training method for a feature extraction model provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an autoencoder model provided in an embodiment of the present invention; Figure 8 for Figure 1 A flowchart illustrating one actual manifestation of S01 in a method for predicting welding quality is provided. Figure 9 This is a schematic diagram of a solder joint marking process provided in an embodiment of the present invention; Figure 10 This is a schematic diagram of a post-weld image acquisition process provided in an embodiment of the present invention; Figure 11 This is a schematic diagram of the structure of a welding quality prediction device provided in an embodiment of the present invention; Figure 12 A flowchart illustrating a training method for a welding quality prediction model provided in an embodiment of the present invention; Figure 13 for Figure 12 A flowchart illustrating an actual representation of S85 in the training method of a welding quality prediction model provided. Figure 14 This is a schematic diagram of the structure of a training device for a welding quality prediction model provided in an embodiment of the present invention. Detailed Implementation

[0023] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0025] The terminology used in this embodiment is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of the invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0026] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0027] Please refer to Figure 1 This is a flowchart illustrating a welding quality prediction method provided in an embodiment of the present invention. The welding quality prediction method provided by the present invention is applied to a welding quality prediction device, which can be a server, a welding machine, or a computer host, etc. The welding quality prediction method of the present invention includes the following steps: Step S01: Based on the post-weld image, extract the solder joints to form a solder joint diagram.

[0028] In this embodiment, the post-weld image is an image obtained by photographing the material after welding. The post-weld image can be output by the welding machine. For example, after the fixture on the welding machine clamps the material, the laser in the welding machine welds the material. After welding is completed, the imaging module in the welding machine photographs the material after welding to form a post-weld image, and outputs the post-weld image.

[0029] In some embodiments, the welding machine can transmit post-weld images to a welding quality prediction device for analysis and storage. Post-weld images can also be obtained from the welding quality prediction device; of course, post-weld images can also be obtained through other means, which are not limited in this embodiment.

[0030] The welding quality prediction device can identify the appearance information of the welded parts of the material in the post-weld image, including the shape, size, color and texture of the weld. By extracting the weld to form a weld map, it can ensure that subsequent feature extraction only reflects the state of the weld itself, thereby establishing an accurate mapping relationship between the weld map and the welding quality prediction value.

[0031] Based on this, the welding quality prediction device can analyze the position and appearance information of the weld points in the post-weld image, extract the weld points from the post-weld image, and form a weld point map.

[0032] Since post-weld images inevitably contain redundant information such as material residue, fixture marks, and uneven lighting, directly using them for feature extraction would introduce a large amount of noise interference, causing the model to fail to accurately capture subtle appearance differences in the weld joints (such as changes in weld width and depth due to defocusing). Algorithmic models (such as object detection models, convolutional neural network models, and support vector machine models) can be used to identify the positional boundaries of one or more weld joints in the post-weld image, thereby separating the weld joints from other interfering elements and forming a weld joint map containing only weld joint information, which can then serve as standardized input for subsequent feature extraction.

[0033] Step S02: Based on the solder joint image, determine the neural network features and image features corresponding to the post-soldering image.

[0034] In this embodiment, after acquiring the weld point image, the welding quality prediction device can analyze the weld point image to extract the image features and neural network features corresponding to the post-weld image. For example, features can be extracted from the weld point image using a pre-trained neural network model (such as a convolutional model, Faster R-CNN model, etc.). The neural network model can map the high-dimensional weld point image to a low-dimensional latent space to obtain low-dimensional neural network features. Specifically, the neural network features include features in dimensions such as weld point fullness, weld point surface granularity, weld point edge steepness, degree of burrs around the weld point, degree of spatter around the weld point, degree of reflectivity of the weld point, weld point symmetry, and weld point texture consistency. Simultaneously, the image features corresponding to the post-weld image are determined based on the image attributes of the weld point image. These image features can include features in dimensions such as weld point diameter, mode of grayscale distribution, grayscale mean, grayscale standard deviation, and texture complexity. Texture complexity includes features in dimensions such as contrast, energy, isomorphism, correlation, and entropy of the weld point.

[0035] The number of dimensions of neural network features can be one or more, and correspondingly, the number of dimensions of image features can also be one or more. In some embodiments, the number of dimensions of neural network features can be set to be the same as the number of dimensions of image features. For example, the number of dimensions of both neural network features and image features can be set to 8 dimensions. This ensures that when the welding quality prediction model makes welding quality predictions based on neural network features and image features, the neural network features and image features can obtain equivalent representational capacity, avoiding the weakening of a feature's weight allocation or information contribution due to dimensional differences. For example, if the number of dimensions of neural network features is greater than that of image features, the welding quality prediction model may over-rely on the multi-dimensional neural network features, while image features, due to insufficient dimensionality, may not be able to fully participate in the decision-making process, ultimately affecting the comprehensiveness and accuracy of welding quality prediction. By maintaining dimensional consistency, the model can more evenly integrate the information of neural network features and image features, improving overall prediction performance.

[0036] Step S03: Input the neural network features and image features into the welding quality prediction model to obtain the welding quality prediction value corresponding to the post-weld image.

[0037] Traditional welding quality inspection methods, such as pull-out force testing, are costly and inefficient, and can only test a small number of samples. Therefore, in this embodiment, neural network features and image features are input into a pre-trained welding quality prediction model. The welding quality prediction model can predict the welding quality of the post-weld material based on the neural network features and image features, and then evaluate whether the welding quality of the post-weld material meets the standards based on the predicted welding quality value, thereby improving the scope and efficiency of welding quality inspection and reducing inspection costs.

[0038] Image features can only provide simple image information for welding quality prediction models, and have significant limitations when dealing with complex conditions such as weld penetration fluctuations caused by defocusing and abnormal energy distribution. Relying solely on neural network features is susceptible to noise interference affecting feature stability. This embodiment addresses the limitation of image features in capturing high-order image features. By utilizing both image and neural network features for welding quality prediction, the accuracy of the welding quality prediction model is effectively improved.

[0039] Based on the above technical solution, the welding quality prediction method provided in this embodiment of the invention extracts weld points from post-weld images to form a weld point map; based on the weld point map, it determines the neural network features and image features corresponding to the post-weld images; and it inputs the neural network features and image features into a welding quality prediction model to obtain the predicted welding quality value corresponding to the post-weld images. This invention analyzes both the neural network features and image features of the post-weld images using a welding quality prediction model to predict the welding quality of the post-weld materials corresponding to the post-weld images, thereby effectively improving the accuracy of the predicted welding quality value and thus enhancing the accuracy of welding quality inspection.

[0040] Meanwhile, based on image analysis and model prediction, this invention can predict the welding quality of all post-weld materials without physically damaging the post-weld materials, without additional testing equipment and material consumption, thus saving the cost of welding quality inspection, reducing manual intervention, lowering labor costs, and improving the efficiency of welding quality determination.

[0041] In some embodiments, step S02, which involves determining the neural network features corresponding to the post-soldering image based on the solder joint image, includes the following steps: Step S11: Based on the feature extraction model, the solder joint image is subjected to dimensionality reduction and feature extraction processing to obtain the neural network features corresponding to the solder joint image.

[0042] In this embodiment, the solder joint image is input into the feature extraction model. The feature extraction model can utilize the multi-layer nonlinear transformation capability of deep neural networks to compress the high-dimensional information of the solder joint image to a low dimension, and perform feature extraction on the solder joint image to obtain the neural network features corresponding to the solder joint image and output them.

[0043] Step S12: Based on the neural network features corresponding to the solder joint image, determine the neural network features corresponding to the post-soldering image.

[0044] In some embodiments, the neural network features corresponding to the post-soldering image can be determined based on the neural network features corresponding to the solder joint image, combined with the number of solder joint images corresponding to the post-soldering image. For example: If there is only one solder joint in the post-soldering image, then there will only be one solder joint image generated based on the post-soldering image. In this case, the neural network features corresponding to the solder joint image will be directly determined as the neural network features corresponding to the post-soldering image.

[0045] If there are multiple solder joints in the post-soldering image, multiple solder joint images can be generated based on the post-soldering image. At this time, the average value of the neural network features corresponding to all solder joint images can be calculated, and the average value can be determined as the neural network feature corresponding to the post-soldering image.

[0046] In this embodiment, after obtaining the neural network features corresponding to the weld point image, the neural network features corresponding to the post-weld image are determined based on the neural network features corresponding to the weld point image, providing a standardized input for the welding quality prediction model. This enables the welding quality prediction model to more accurately identify abnormal changes in the weld point image when predicting welding quality, thereby improving the accuracy of the welding quality prediction value.

[0047] In some embodiments, the step S11 mentioned above, which involves performing dimensionality reduction and feature extraction on the solder joint image based on the feature extraction model to obtain neural network features, may specifically include the following steps: Step S21: Project the solder joint image onto the feature vector space of the feature extraction model to obtain the feature values ​​of the solder joint image.

[0048] Step S22: Determine the feature values ​​of the solder joint image as the neural network features corresponding to the solder joint image.

[0049] In this embodiment, by utilizing a feature extraction model, the solder joint image is mapped from the original pixel space to the feature vector space, generating a set of fixed-length numerical feature values, i.e., the feature values ​​of the solder joint image.

[0050] In some embodiments, the feature vector space of the feature extraction model may be multiple, and its number may be the same as the number of dimensions of the neural network features.

[0051] For example, neural network features can include eight dimensions of features, such as solder joint fullness, solder joint surface graininess, solder joint edge steepness, solder joint burr degree, solder joint spatter degree, solder joint reflectivity, solder joint symmetry, and solder joint texture consistency. The corresponding eight-dimensional neural network features can be obtained by projecting the solder joint image onto the feature vector space corresponding to each dimension of the feature.

[0052] In some embodiments, the post-soldering image may have one or more solder joints. As mentioned in S02, determining the image features corresponding to the post-soldering image based on the solder joint image may specifically include the following steps: S021, Based on the solder joint diagram, determine the width and height of the solder joints.

[0053] S022, the diameter of the weld point is determined as the image feature corresponding to the post-weld image.

[0054] In some embodiments, please refer to Figure 2 ,for Figure 1 The flowchart illustrates the first practical manifestation of S02 in the provided method for predicting welding quality. If the post-weld image has multiple weld points, as mentioned in step S02, based on the weld point map, the image features corresponding to the post-weld image are determined, which may specifically include, for example... Figure 2 The steps shown are as follows: Step S31: Based on the solder joint diagram, determine the total number of solder joints and the width and height of each solder joint.

[0055] In this embodiment, after extracting solder joints and forming solder joint images based on post-weld images, the number of solder joint images can be counted to determine the total number of solder joints, and the width and height of each solder joint image can be obtained and determined as the width and height of the corresponding solder joint.

[0056] Step S32: Determine the diameter of the solder joint based on the total number of solder joints and the width and height of each solder joint.

[0057] In this embodiment, after obtaining the number of all solder joints and the width and height of each solder joint, the diameter of the solder joint can be calculated based on the width and height of all solder joints.

[0058] For example, a welding quality prediction device can calculate the equivalent diameter of a single weld point based on the independent width and height data of each weld point. For instance, the average width and height can be taken as the equivalent diameter of the weld point. Then, the individual diameters of all weld points on the post-weld image are arithmetically averaged to obtain the final single weld point diameter feature.

[0059] In some embodiments, the solder joint diameter is calculated using formula (1): (1) in, The diameter of the solder joint. This represents the total number of solder joints in the post-weld image. Let be the width of the i-th solder joint image. Let be the height of the i-th solder joint diagram.

[0060] In this embodiment, the solder joint diameter is obtained by averaging the width and height of each solder joint image, summing the average values ​​of all solder joints, and then averaging the sums again. The solder joint diameter quantifies the size of the solder joint, and the size of the solder joint directly affects its strength and reliability.

[0061] In some embodiments, if the post-weld image has only one solder joint, the width and height of the solder joint can also be determined, and the average of the width and height of the solder joint is determined as the solder joint diameter.

[0062] Step S33: Determine the diameter of the solder joint as the image feature corresponding to the post-weld image.

[0063] In this embodiment, determining the weld joint diameter as the image feature corresponding to the post-weld image allows the welding quality prediction model to consider the strength and reliability of the weld joint when predicting welding quality, thereby improving the accuracy of the predicted welding quality value.

[0064] The image features in this embodiment of the invention are not limited to the solder joint diameter; image features may also include the mode of grayscale distribution. Please refer to... Figure 3 ,for Figure 1 A flowchart illustrating the second actual manifestation of S02 in a method for predicting welding quality.

[0065] In some embodiments, the step S02, which involves determining the image features corresponding to the post-soldering image based on the solder joint diagram, may specifically include, for example: Figure 3 The steps shown are as follows: Step S41: Fit the grayscale value of each pixel in the solder joint image using a preset probability density function to obtain the scale parameter and shape parameter of the preset probability density function.

[0066] In welding quality inspection, the grayscale distribution of weld joints reflects the uniformity and density of the weld surface, and is an important characteristic for evaluating welding quality. Traditional grayscale statistical methods can only provide simple grayscale mean or variance, making it difficult to fully capture the details of the grayscale distribution, especially when dealing with complex weld surfaces, where key information is easily lost.

[0067] Therefore, in this embodiment, the welding quality prediction device traverses the gray intensity value (usually a discrete integer from 0 to 255) of each effective pixel in the weld point image, treats it as a random sample set, and uses maximum likelihood estimation or least squares method to fit it to a preset probability density function, thereby obtaining the scale parameter (i.e., the degree or range of gray distribution) and shape parameter (i.e., the skewness, kurtosis and other morphological characteristics of gray distribution) of the preset probability density function to describe the distribution law of gray values ​​in the weld point image.

[0068] In some embodiments, the preset probability density function can be a Weiper distribution probability density function. Using a Weiper distribution for fitting can more accurately describe the characteristics of the gray-scale distribution, provide richer statistical information, and improve detection accuracy.

[0069] It should be noted that the above embodiments are only one preferred embodiment of the present invention, and the specific category of the preset probability density function is not limited in this embodiment.

[0070] Step S42: Determine the mode of grayscale distribution of the solder joint image based on the scale parameter and shape parameter of the preset probability density function.

[0071] In this embodiment, the mode of grayscale distribution of the solder joint image is determined based on the scale parameter and shape parameter of the preset probability density function. The mode of grayscale distribution is the grayscale value with the highest frequency of occurrence, which can reflect the central tendency of grayscale distribution of the solder joint image.

[0072] For example, the mode of the gray-scale distribution can be calculated using formula (2): (2) in, The mode of the grayscale distribution. For scale parameters, For shape parameters.

[0073] Step S43: Determine the image features corresponding to the post-weld image based on the mode of the grayscale distribution of the solder joint image.

[0074] In some embodiments, the image features corresponding to the post-soldering image can be determined based on the mode of the grayscale distribution of the solder joint image and the number of solder joint images corresponding to the post-soldering image. For example: If the number of solder joint images corresponding to the post-weld image is one, that is, there is only one solder joint in the post-weld image, then the mode of the grayscale distribution of the solder joint image can be directly determined as the image feature corresponding to the post-weld image.

[0075] If the number of solder joint images corresponding to the post-weld image is greater than or equal to two, that is, there are multiple solder joints in the post-weld image, then the average value of the mode of the grayscale distribution of all solder joint images can be calculated, and this average value can be determined as the image feature corresponding to the post-weld image.

[0076] In this embodiment, after obtaining the mode of the grayscale distribution of the weld point image, the image features corresponding to the post-weld image can be determined based on the mode of the grayscale distribution of the weld point image, providing a standardized input for the welding quality prediction model. This enables the welding quality prediction model to more sensitively capture subtle changes in the weld point image when predicting welding quality, thereby improving the accuracy of the welding quality prediction value.

[0077] In another embodiment, the image features may further include the grayscale standard deviation. Step S41, after fitting the grayscale value of each pixel in the solder joint image using a preset probability density function to obtain the scale parameter and shape parameter of the preset probability density function, may further include the following steps: Step S44: Determine the grayscale standard deviation of the solder joint image based on the scale parameter and shape parameter of the preset probability density function.

[0078] In this embodiment, after obtaining the scale and shape parameters of the preset probability density function, the grayscale standard deviation is determined based on these parameters. The grayscale standard deviation reflects the distribution of grayscale values ​​in the solder joint image, providing information about the surface uniformity of the solder joint. For example, a high grayscale standard deviation indicates the presence of non-uniformity or defects on the solder joint surface, such as holes or cracks, while a low grayscale standard deviation indicates a more uniform solder joint surface and better quality.

[0079] For example, the standard deviation of grayscale can be calculated using formula (3): (3) in, The standard deviation of grayscale It is the gamma function, and , where n is a positive integer. For scale parameters, For shape parameters.

[0080] Step S45: Determine the image features corresponding to the post-weld image based on the grayscale standard deviation of the solder joint image.

[0081] In some embodiments, the image features corresponding to the post-soldering image can be determined based on the grayscale standard deviation of the solder joint image and the number of solder joint images corresponding to the post-soldering image. For example: If the number of solder joint images corresponding to the post-weld image is one, that is, there is only one solder joint in the post-weld image, then the grayscale standard deviation of the solder joint image can be directly determined as the image feature corresponding to the post-weld image.

[0082] If the number of solder joint images corresponding to the post-weld image is greater than or equal to two, that is, there are multiple solder joints in the post-weld image, then the average grayscale standard deviation of all solder joint images can be calculated, and this average value can be determined as the image feature corresponding to the post-weld image.

[0083] In this embodiment, after obtaining the grayscale standard deviation of the weld point image, the image features corresponding to the post-weld image can be determined based on the grayscale standard deviation of the weld point image, providing a standardized input for the welding quality prediction model. This enables the welding quality prediction model to capture information on the surface uniformity of the weld point when predicting welding quality, thereby improving the accuracy of the welding quality prediction value.

[0084] In another embodiment, image features may also include texture complexity.

[0085] Please refer to Figure 4 ,for Figure 1 A flowchart illustrating the third actual manifestation of S02 in a method for predicting welding quality.

[0086] In some embodiments, the step S02, which involves determining the image features corresponding to the post-soldering image based on the solder joint diagram, may specifically include, for example: Figure 4 The steps shown are as follows: Step S51: Determine the gray-level co-occurrence matrix of the solder joint image based on the gray-level co-occurrence matrix algorithm.

[0087] In this embodiment, by using the gray-level co-occurrence matrix algorithm to calculate the gray values ​​of pixel pairs and their spatial position relationships in the solder joint image, a gray-level co-occurrence matrix is ​​generated to describe the spatial distribution relationship of pixel gray values ​​in the solder joint image, which can describe rich information about the texture of the solder joint image.

[0088] For example, the gray-level co-occurrence matrix can be calculated using formula (4): (4) in, The grayscale co-occurrence matrix of the solder joint image. The first pixel in the pixel pair in the solder joint image grayscale value, The second pixel in the pixel pair in the solder joint image grayscale value, The distance between each pixel pair in the solder joint image. This refers to the grayscale level of the solder joint image. As direction, For direction The unit vector.

[0089] Step S52: Determine the texture complexity of the solder joint map based on the gray-level co-occurrence matrix.

[0090] In this embodiment, texture complexity reflects the complexity and regularity of grayscale value changes in the solder joint image, providing more image detail information than a single grayscale statistic. By quantifying texture complexity, potential defects in the solder joint image can be identified and evaluated, thereby improving the accuracy and reliability of welding quality inspection. Based on this, after obtaining the grayscale co-occurrence matrix, the texture complexity that characterizes the texture complexity of the solder joint image can be calculated by analyzing the data in the grayscale co-occurrence matrix.

[0091] In some embodiments, texture complexity may include contrast, energy, isomorphism, correlation, and entropy, wherein: Contrast can be calculated using formula (5): (5) in, For contrast; Energy can be calculated using formula (6): (6) in, For energy; Isomorphism can be calculated using formula (7): (7) in, They are isomorphic; Correlation can be calculated using formula (8): (8) in, For correlation, This represents the average of the x-coordinates of the pixels in the solder joint image. This represents the mean of the vertical coordinates of the pixels in the solder joint image. The standard deviation of the x-coordinate of the pixels in the solder joint image. The standard deviation of the ordinate of the pixels in the solder joint image; Entropy can be calculated using formula (9): (9) in, Entropy.

[0092] Step S53: Determine the image features corresponding to the post-soldering image based on the texture complexity of the solder joint image.

[0093] In some embodiments, the image features corresponding to the post-soldering image can be determined based on the texture complexity of the solder joint image and the number of solder joint images corresponding to the post-soldering image. For example: If the number of solder joint images corresponding to the post-weld image is one, that is, there is only one solder joint in the post-weld image, then the texture complexity of the solder joint image can be directly determined as the image feature corresponding to the post-weld image.

[0094] If the number of solder joint images corresponding to the post-soldering image is greater than or equal to two, that is, there are multiple solder joints in the post-soldering image, then the average texture complexity of all solder joint images can be calculated, and this average value can be determined as the image feature corresponding to the post-soldering image.

[0095] In this embodiment, after obtaining the texture complexity of the weld point image, the image features corresponding to the post-weld image can be determined based on the texture complexity of the weld point image, providing a standardized input for the welding quality prediction model. This enables the welding quality prediction model to more sensitively capture the changes in texture in the weld point image when predicting welding quality, thereby more accurately identifying welding defects and improving the accuracy of welding quality prediction values.

[0096] Please refer to Figure 5 ,for Figure 1 A flowchart illustrating one practical manifestation of S03 in a method for predicting welding quality.

[0097] Based on the above embodiments, in some embodiments, the welding quality prediction model can be a linear regression model or a machine learning model. Optionally, the welding quality prediction model is a Gaussian process regression model. The predicted welding quality values ​​include the predicted pull-out force, the reliability of the predicted pull-out force, the quality category (e.g., the appearance quality and internal quality of the weld), the welding reliability level (e.g., level one, level two, level three, etc.), and the welding reliability index. For ease of understanding, the predicted welding quality values ​​are illustrated using the predicted pull-out force and the reliability of the predicted pull-out force as examples.

[0098] Based on this, as mentioned in step S03, the neural network features and image features are input into the welding quality prediction model to obtain the predicted welding quality value corresponding to the post-weld image. Specifically, this may include, for example... Figure 5 The steps shown are as follows: Step S61: Standardize the neural network features and image features to obtain the input values.

[0099] In this embodiment, the predicted welding quality value is obtained by the welding quality prediction model based on the neural network features and image features of the post-weld image. After obtaining the neural network features and image features of the post-weld image, they need to be input into the welding quality prediction model. However, neural network features and image features are different types of feature data. Before being input into the welding quality prediction model, the neural network features and image features need to be converted into data that the welding quality prediction model can recognize. Therefore, the neural network features and image features can be standardized to map them to the same numerical range, resulting in a standardized joint input vector, i.e., the input value.

[0100] Optionally, the neural network features may include eight dimensions of features, such as solder joint fullness, solder joint surface graininess, solder joint edge steepness, solder joint burr degree, solder joint spatter degree, solder joint reflectivity, solder joint symmetry, and solder joint texture consistency. The image features may include eight dimensions of features, such as solder joint diameter, gray level distribution mode, gray level standard deviation, and texture complexity, such as contrast, energy, isomorphism, correlation, and entropy.

[0101] For example, eight neural network features, such as solder joint fullness, solder joint surface graininess, solder joint edge steepness, solder joint burr degree, solder joint spatter degree, solder joint reflectivity, solder joint symmetry, and solder joint texture consistency, are linearly transformed (e.g., z-score normalization or min-max normalization) with eight image features, such as solder joint diameter, gray-level distribution mode, gray-level standard deviation, and texture complexity (contrast, energy, isomorphism, correlation, and entropy). Then, all the features after the linear transformation are concatenated to obtain the feature vector, i.e., the input value, which can be directly used for Gaussian process regression model calculation.

[0102] Step S62: Calculate the mean and variance of the input values ​​using the kernel function of the Gaussian process regression model.

[0103] In this embodiment, after the input values ​​are input into the Gaussian process regression model, the Gaussian process regression model can calculate the nonlinear similarity between the input values ​​and the training samples in the feature space using a kernel function. This allows the model to derive the posterior probability distribution of the predicted values, calculate the mean and variance of this posterior probability distribution, and use these as the mean and variance of the input values ​​for output. The training samples are those stored during the training phase of the Gaussian process regression model.

[0104] In some embodiments, assuming the kernel function of the Gaussian process regression model is a radial basis function (RBF), the mean can be calculated using formula (10) based on the input values: (10) in, For input values The distance to each training sample, i.e. , is a 1×n vector; For hyperparameters; It is a unit vector; The pull-out force test value corresponding to the training sample, i.e. .

[0105] Calculate the variance of the input values ​​according to formula (11): (11) in, Prior variance refers to the variance of the input values ​​without any prior data. Uncertainty in point prediction. For each training sample and input value The distance, i.e. ,for The vector.

[0106] Step S63: The mean is determined as the predicted welding quality value corresponding to the post-weld image, and the confidence level of the predicted welding quality value is determined based on the variance.

[0107] In this embodiment, the Gaussian process regression model can output the mean and variance of the input values. The mean represents the welding quality prediction value corresponding to the post-weld image that the Gaussian process regression model is most likely to obtain based on the neural network features and image features corresponding to the post-weld image. The variance represents the uncertainty of the welding quality prediction value. The larger the variance, the higher the uncertainty of the welding quality prediction value and the lower the reliability; the smaller the variance, the lower the uncertainty of the welding quality prediction value and the higher the reliability.

[0108] Based on the above technical solution, this embodiment maps neural network features and image features to the probabilistic framework of Gaussian process regression through standardization processing and kernel function calculation. This not only outputs predicted welding quality values ​​for evaluating welding quality but also outputs the variance used to quantify the reliability of the predicted welding quality values, providing a more accurate basis for welding quality prediction. Simultaneously, based on the nonlinear mapping capability of the kernel function, the Gaussian process regression model exhibits extremely strong learning robustness in small sample data scenarios, avoiding the dependence of deep supervised models on large amounts of labeled data.

[0109] Please refer to Figure 6 The flowchart below shows a training method for a feature extraction model provided in an embodiment of the present invention, including the following steps: Step S71: Based on historical post-soldering images, extract historical solder joints to form a historical solder joint map.

[0110] In this embodiment, historical post-weld images can be obtained. By automatically locating and cropping the solder joint area from the historical post-weld images, a historical solder joint map for training the feature extraction model can be generated. The process is the same as step S01 and will not be described again here.

[0111] Step S72: Project the historical solder joint image onto the feature vector space to obtain the feature values.

[0112] In this embodiment, after obtaining the historical solder joint image, the initial feature extraction model can be trained using the historical solder joint image. At this time, the historical solder joint image can be projected onto the feature vector space of the initial feature extraction model to obtain feature values.

[0113] In this embodiment, features of historical solder joint images are extracted using an initial feature extraction model and then compressed into fixed-length feature values. This process achieves a nonlinear transformation from image space to feature space, providing a data foundation for subsequent image reconstruction and difference calculation.

[0114] In some embodiments, the initial feature extraction model can be a pre-set autoencoder model containing an encoder and a decoder, for reference. Figure 7 As shown.

[0115] Step S72 can be implemented using the encoder in the autoencoder model. The encoder consists of multiple stacked convolutional and pooling layers, with a fully connected layer at the end to output multi-dimensional feature values. The encoder projects the historical solder joint image onto multiple feature vector spaces, using the feature values... A diagram representing historical solder joints.

[0116] Step S73: Restore the eigenvalues ​​in the eigenvector space to obtain the reconstructed image.

[0117] In this embodiment, an initial feature extraction model is used to restore the feature values ​​in the feature vector space, generating a reconstructed image with the same size as the historical solder joint image. The initial feature extraction model gradually expands the spatial resolution of the feature map and restores the channel information through a multi-layer transposed convolutional neural network and upsampling operations, ultimately outputting a reconstructed image with the same size as the historical solder joint image.

[0118] In some embodiments, the implementation of step S73 may rely on Figure 7 The decoder in the autoencoder model shown consists of convolutional layers that are mirror images of the encoder. The input is a multidimensional feature vector, which is expanded by fully connected layers and then progressively upsampled to the target resolution (e.g., 224×224 pixels) through 4-5 layers of transposed convolutions. The decoder restores the feature values ​​in the feature vector space to obtain the reconstructed image.

[0119] Step S74: Determine the degree of difference between the reconstructed image and the historical solder joint image, and train the model based on the degree of difference to form a feature extraction model.

[0120] In this embodiment, after generating the reconstructed image, the difference between the reconstructed image and the historical solder joint image can be calculated to obtain the difference degree between the reconstructed image and the historical solder joint image. The difference degree is then checked to see if it is below a preset threshold. If the difference degree is greater than the preset threshold, it indicates that the difference between the current reconstructed image and the historical solder joint image is too large. In this case, the parameters of each layer in the initial feature extraction model can be adjusted based on the difference degree as a loss signal.

[0121] After adjusting the parameters of each layer, steps S71 to S74 can be re-executed until the difference converges to below the preset threshold. In this way, the initial feature extraction model corresponding to the difference being less than the preset threshold can be used as the feature extraction model, thereby obtaining a feature extraction model with stable feature extraction capabilities.

[0122] Based on the above technical solution, this embodiment trains the feature extraction model through a self-supervised reconstruction mechanism, which can automatically complete the model learning with only historical solder joint images, reducing manual annotation and improving manual efficiency.

[0123] Based on the above embodiments, in some embodiments, the step S01 mentioned, extracting solder joints based on the post-weld image to form a solder joint map, can be specifically implemented using an object detection model.

[0124] In this embodiment, the object detection model is a trained deep learning model that can analyze the input post-weld image, identify the solder joints in the image, and form a solder joint map, laying the foundation for subsequent feature extraction and quality assessment.

[0125] Please refer to Figure 8 ,for Figure 1 A flowchart illustrating one practical manifestation of S01 in a provided method for predicting welding quality. For example... Figure 8 As shown, an object detection model is used to extract solder joints from post-weld images and form a solder joint map. The specific process may include, for example: Figure 8 The steps shown are as follows: Step S81: Process the post-weld images to obtain a first number of post-weld images of different sizes.

[0126] Specifically, in the object detection model, a group of multiple convolutional layers (e.g., C1, C2, C3, C4, C5 convolutional layers) processes the post-weld image to obtain a first number of post-weld images of different sizes (e.g., 5 sizes of post-weld images). The first number can be the same as the number of convolutional layers; for example, there are 5 convolutional layers, namely C1, C2, C3, C4, C5, C6, C7, C8, C9 ... 5， The quantity can be 5 types, that is, the number of post-weld images of different sizes is 5 different sizes of post-weld images.

[0127] Step S82: From the first number of post-weld images of different sizes, select a second number of post-weld images of different sizes, perform upsampling and feature fusion, and obtain a second number of post-weld feature images of different sizes.

[0128] In this embodiment, the second quantity is less than the first quantity. For example, the second quantity is 5 types and the first quantity is 3 types. For instance, if there are 5 different sizes of post-weld images, then 3 different sizes of post-weld images output by convolutional layers C3, C4, and C5 are selected. Then, based on the 3 different sizes of post-weld images, adjacent post-weld images are upsampled and feature fused to obtain the second quantity of post-weld feature images of different sizes, so as to highlight the solder joint features in the post-weld images and facilitate the subsequent identification of solder joints. Specifically, the post-weld image output by C5 is upsampled and feature fused to obtain the first size of post-weld feature image P5. Then, the post-weld image output by C4 and P5 are upsampled and feature fused to obtain the second size of post-weld feature image P4. Then, the post-weld image output by C3 and P4 are upsampled and feature fused to obtain the third size of post-weld feature image P3.

[0129] Step S83: Perform feature prediction processing on the second number of post-weld feature images of different sizes to obtain multiple prediction boxes. Each prediction box carries three dimension values, confidence level, and target type.

[0130] In this embodiment, the target type may include fixture features, solder joint features, material features, etc. The target type in this embodiment is solder joint features; solder joint features refer to the solder joint grid in the post-weld feature image; the three dimensions refer to the width, height, and center point coordinates of each prediction box, or the coordinates of the two opposite corners and the center point coordinates of each prediction box; confidence level refers to the probability that the solder joint feature prediction box contains solder joint features.

[0131] Step S84: Based on real solder joint samples, each prediction box carries three dimension values, confidence level, and target category, determine the prediction box containing solder joint features from multiple prediction boxes.

[0132] Step S85: Determine the solder joint map based on the prediction box containing solder joint features and its three dimensional values.

[0133] In some embodiments, before using the object detection model to determine the solder joint image in the post-weld image, the object detection model can be trained based on the post-weld image data with labeled solder joints. For example, please refer to... Figure 9 This is a schematic diagram of a solder joint marking process provided by an embodiment of the present invention. During the solder joint marking process, solder joints in the post-soldering image can be marked using methods such as machine recognition or manual annotation.

[0134] Please refer to Figure 10 This is a schematic diagram of a post-weld image acquisition process provided in an embodiment of the present invention.

[0135] like Figure 10As shown, in some embodiments, due to the differences between different laser welding machines, the corresponding processing information can be collected simultaneously when acquiring post-weld images and uploaded to a server for unified management. The server stores post-weld images and historical post-weld images, and sends these stored images to a welding quality prediction device for training various models and predicting welding quality values.

[0136] This processing information can include the welding machine code, enabling customized model training based on the specific characteristics and welding parameters of each welding machine. This allows for the configuration of a corresponding welding quality prediction model for each machine, thereby improving the accuracy and adaptability of predictions. This targeted modeling method can effectively address the welding quality differences between different machines caused by variations in equipment, process parameters, and material properties. It enables precise monitoring and quality prediction of the welding process for each machine, ultimately enhancing the stability and reliability of overall welding production.

[0137] Furthermore, the processing information may also include material codes, fixture information, project information, workstation information, etc., so that after obtaining the welding quality prediction value based on the neural network features and image features corresponding to the post-weld image using the welding quality prediction model, the system can analyze the links that may have problems based on the processing information and welding quality prediction value. For example, if a certain link leads to a low welding quality prediction value, corresponding improvement measures can be generated accordingly.

[0138] Please refer to Figure 11 This is a schematic diagram of the structure of a welding quality prediction device provided in an embodiment of the present invention. Figure 11 As shown, the welding quality prediction device may include: Solder joint extraction module 100 is used to extract solder joints based on post-weld images and form a solder joint image; Feature module 200 is used to determine the neural network features and image features corresponding to the post-weld image based on the solder joint image; The prediction module 300 is used to input neural network features and image features into the welding quality prediction model to obtain the welding quality prediction value corresponding to the post-weld image.

[0139] Based on the above embodiments, in some embodiments, the feature module 200 may specifically be used for: Based on the feature extraction model, the solder joint image is subjected to dimensionality reduction and feature extraction to obtain the neural network features corresponding to the solder joint image. Based on the neural network features corresponding to the solder joint image, the neural network features corresponding to the post-soldering image are determined.

[0140] Based on the above embodiments, in some embodiments, the feature module 200 may specifically be used for: Based on historical post-weld images, historical solder joints are extracted to form a historical solder joint map; Projecting the historical solder joint image onto the feature vector space yields the feature values; The eigenvalues ​​in the eigenvector space are restored to obtain the reconstructed image; The degree of difference between the reconstructed image and the historical solder joint image is determined, and the model is trained based on the degree of difference to form a feature extraction model.

[0141] Based on the above embodiments, in some embodiments, the feature module 200 may specifically be used for: The solder joint image is projected onto the feature vector space of the feature extraction model to obtain the feature values ​​of the solder joint image; The feature values ​​of the solder joint image are determined as the neural network features corresponding to the solder joint image.

[0142] Based on the above embodiments, in some embodiments, the post-weld image has multiple solder joints; Feature module 200 can be specifically used for: Based on the solder joint diagram, determine the total number of solder joints and the width and height of each solder joint; The diameter of a solder joint is determined based on the total number of solder joints and the width and height of each solder joint. The diameter of the weld point is determined as the image feature corresponding to the post-weld image.

[0143] Based on the above embodiments, in some embodiments, the feature module 200 may specifically be used for: The grayscale value of each pixel in the solder joint image is fitted using a preset probability density function to obtain the scale parameter and shape parameter of the preset probability density function. The mode of grayscale distribution of the solder joint image is determined based on the scale and shape parameters of the preset probability density function. The image features corresponding to the post-weld image are determined based on the mode of the grayscale distribution of the solder joint image.

[0144] Based on the above embodiments, in some embodiments, the feature module 200 can also be used for: The grayscale standard deviation of the solder joint image is determined based on the scale and shape parameters of the preset probability density function. Image features corresponding to the post-weld image are determined based on the grayscale standard deviation of the solder joint image.

[0145] Based on the above embodiments, in some embodiments, the feature module 200 may specifically be used for: The gray-level co-occurrence matrix of the solder joint image is determined based on the gray-level co-occurrence matrix algorithm. The texture complexity of the solder joint map is determined based on the gray-level co-occurrence matrix; The image features corresponding to the post-soldering image are determined based on the texture complexity of the solder joint image.

[0146] Based on the above embodiments, in some embodiments, the predicted welding quality value includes the predicted pull-out force value and the confidence level of the predicted pull-out force value, and the predicted welding quality model includes a Gaussian process regression model. The prediction module 300 can be specifically used for: The neural network features and image features are standardized to obtain the input values; The mean and variance of the input values ​​are calculated using the kernel function of the Gaussian process regression model. The mean value is determined as the predicted welding quality value corresponding to the post-weld image, and the confidence level of the predicted welding quality value is determined based on the variance.

[0147] Please refer to Figure 12 The flowchart below shows a training method for a welding quality prediction model provided in an embodiment of the present invention, which includes the following steps: Step S91: Based on the post-weld image, extract the solder joints to form a training solder joint image.

[0148] In this embodiment, the weld points are extracted from the post-weld image to form a training weld point map for training the welding quality prediction model. The process is the same as step S01 and will not be repeated here.

[0149] Step S92: Based on the training solder joint image, determine the neural network features and image features.

[0150] In this embodiment, neural network features and image features are extracted from the training weld point diagram to train the welding quality prediction model. The process is the same as step S02 and will not be repeated here.

[0151] Step S93: Based on the pull-out force test value corresponding to the training solder joint image, and combined with the neural network features and image features of the training solder joint image, a training sample is formed.

[0152] In this embodiment, training samples for training an initial weld quality prediction model are constructed by pairing the neural network features and image features of the trained weld point map with the pull-out force test values ​​obtained through destructive detection.

[0153] Since neural network features and image features are different types of feature data, they can be converted into data that the welding quality prediction model can recognize before pairing the neural network features and image features of the trained weld spot diagram with the pull-out force test values ​​obtained through destructive testing.

[0154] As can be seen from the above embodiments, the neural network features include at least eight features such as solder joint fullness, solder joint surface graininess, solder joint edge steepness, solder joint burr degree, solder joint spatter degree, solder joint reflectivity, solder joint symmetry, and solder joint texture consistency. The neural network features include at least eight features such as solder joint diameter, gray level distribution mode, gray level standard deviation, and texture complexity including contrast, energy, isomorphism, correlation, and entropy.

[0155] For example, eight neural network features, such as solder joint fullness, solder joint surface graininess, solder joint edge steepness, solder joint burr degree, solder joint spatter degree, solder joint reflectivity, solder joint symmetry, and solder joint texture consistency, are linearly transformed (e.g., z-score normalization or min-max normalization) with eight image features, such as solder joint diameter, gray-level distribution mode, gray-level standard deviation, and texture complexity (contrast, energy, isomorphism, correlation, and entropy). Then, all the features after the linear transformation are concatenated to obtain a feature vector that can be directly used for training a Gaussian process regression model, i.e., a fused feature vector.

[0156] Then, a pull-out force test is performed on the welded material corresponding to the welded image. The actual pull-out force test value is used as a label and bound to the fusion feature vector of the training weld point image corresponding to the welded image to form a training sample with a one-to-one correspondence between "fusion feature vector - pull-out force test value".

[0157] Step S94: Construct a training sample set using the training samples corresponding to each training solder joint diagram.

[0158] In this embodiment, a complete training sample set that can be used for batch training of the model is constructed by systematically aggregating the training samples corresponding to each training solder joint diagram.

[0159] Step S95: Train the initial welding quality prediction model using the training sample set, and use the initial welding quality prediction model after the iteration ends as the welding quality prediction model.

[0160] In this embodiment, the initial welding quality prediction model is optimized for parameters and trained iteratively for hyperparameters using a training sample set until the objective function converges or reaches a preset number of iterations. Finally, the model structure and parameters are solidified to form a welding quality prediction model that can predict welding quality values ​​based on post-weld images.

[0161] In this embodiment, the initial welding quality prediction model can be a decision tree model, a deep neural network model, an ensemble learning model, etc., and this embodiment does not make specific limitations on it.

[0162] Based on the above technical solution, this embodiment provides a training basis for training the welding quality prediction model by constructing training samples. Simultaneously, the model training process achieves collaborative optimization of multimodal information, enabling the model to simultaneously predict welding quality based on neural network features and image features, thereby improving prediction accuracy.

[0163] Based on the above embodiments, in some embodiments, the initial welding quality prediction model can be a Gaussian process regression model.

[0164] In this embodiment, Gaussian process regression is a non-parametric, probabilistic regression method, defined as:

[0165] in, This is the mean function (usually set to 0, which standardizes the data). The kernel function (e.g., the covariance function) is used to measure... and The similarity.

[0166] By defining a Gaussian process, and using the fused feature vectors in the training samples and the corresponding pull-out force test value labels, the mapping relationship between the two is learned, thereby constructing a welding quality prediction model that can predict welding quality values.

[0167] Based on this, as mentioned in step S95, training the initial welding quality prediction model using a training sample set can specifically include, for example: Figure 13 The steps shown are as follows: Step S101: Standardize all training samples in the training sample set.

[0168] In this embodiment, before training the Gaussian process regression model, the neural network features and image features of all samples in the entire training sample set are uniformly standardized, and each feature dimension is linearly transformed to a standard scale according to the statistical parameters of the training set.

[0169] For example, the pull-out force test values ​​in the training samples can be mean-zeroed to ensure that a single feature or predicted value will not affect the overall distribution due to its large range during training. The mean-zeroing of the fused feature vector can be achieved using formula (12): (12) in, Let f be the feature value of the i-th training sample. Let f be the mean of the feature f in the training samples. denoted as the standard deviation of feature f in the training samples.

[0170] The average value of the pull-out force can be zeroed out using formula (13): (13) in, Let be the pull-out force test value of the i-th training sample.

[0171] Step S102: Determine the kernel function of the Gaussian process regression model.

[0172] In this embodiment, a suitable kernel function is selected and defined for the initial Gaussian process regression model. The kernel function is used to calculate the similarity between input feature vectors. The welding quality prediction device needs to determine the type of kernel function (such as the squared exponential kernel, Matérn kernel, etc.) and initialize the hyperparameters of the kernel function (such as length scale, signal variance). The kernel function maps any two sample points in the input space to covariance values, forming a kernel matrix, thereby driving the Gaussian process regression model to predict the pull-out force test value.

[0173] In some embodiments, the kernel function of the Gaussian process regression model mentioned in step S102 can specifically be: S111, determine whether the kernel function selection type has been received from user input.

[0174] If yes, proceed to step S112; otherwise, proceed to step S113.

[0175] S112, Based on the kernel function selection type, determine the kernel function of the Gaussian process regression model.

[0176] S113, the kernel function of the Gaussian process regression model based on the preset kernel function type.

[0177] This embodiment provides a user-interventionable kernel function selection mechanism to enhance the adaptability of the welding quality prediction model to different welding process characteristics. Specifically, users can manually specify the kernel function type according to specific defect patterns (such as texture disorder caused by defocusing), thereby achieving accurate optimization of the welding quality prediction model's performance. At the same time, the preset default kernel function ensures that the welding quality prediction device can run automatically without human intervention, reducing the operational threshold and deployment cost.

[0178] Step S103: Based on the standardized training samples, optimize the hyperparameters of the kernel function by maximizing the marginal likelihood function to form a welding quality prediction model.

[0179] In this embodiment, based on the standardized training samples, the hyperparameters of the kernel function are used as optimization variables. By solving the problem of maximizing the marginal likelihood function, the optimal combination of hyperparameters is automatically found, thereby forming a welding quality prediction model with predictive capabilities.

[0180] In some embodiments, assuming the kernel function of the Gaussian process regression model is a radial basis function, the distance between two sets of training samples can be calculated using formula (14): (14) in, There are two sets of training samples; The variation amplitude of the control function output is represented as a noise term; The length scale determines the smoothness of the function. The larger the value, the more correlated points are considered, and the smoother the function.

[0181] If there are n input values, then an n×n distance matrix K can be obtained:

[0182] Based on this, the hyperparameters of the kernel function can be optimized by maximizing the marginal likelihood function using formula (15): (15) in, For hyperparameters; It is a unit vector; The predicted welding quality value corresponding to the training sample, i.e. , This is the data fit term, used to measure how well the model fits the data. As a penalty for model complexity, It is a constant term. In order to make the result a standard probability distribution, the fitting process can be ignored since there are no hyperparameters.

[0183] Maximizing this function means optimizing its hyperparameters. Hyperparameters in K , We can first initialize a set of guessed hyperparameter combinations, and then use gradient optimization to find the maximum value of this function, thereby optimizing the hyperparameters. After obtaining the optimal hyperparameter combination, a welding quality prediction model is formed.

[0184] Please refer to Figure 14 This is a schematic diagram of the structure of a training device for a welding quality prediction model provided in an embodiment of the present invention. Figure 14 As shown, the training device for this welding quality prediction model may include: The training solder joint image forming module 400 is used to extract solder joints and form a training solder joint image based on the post-weld image. Feature Determination Module 500 is used to determine neural network features and image features based on the trained solder joint map; The sample formation module 600 is used to form training samples based on the pull-out force test value corresponding to the training solder joint image, combined with the neural network features and image features of the training solder joint image; The sample set construction module 700 is used to construct a training sample set using the training samples corresponding to each training solder point image; Training module 800 is used to train the initial welding quality prediction model using the training sample set, and the initial welding quality prediction model after the iteration ends is used as the welding quality prediction model.

[0185] Based on the above embodiments, in some embodiments, the initial welding quality prediction model is a Gaussian process regression model; Training module 800 can be specifically used for: Standardize all training samples in the training sample set; Determine the kernel function of the Gaussian process regression model; Based on the standardized training samples, the hyperparameters of the kernel function are optimized by maximizing the marginal likelihood function to form a welding quality prediction model.

[0186] Based on the above embodiments, in some embodiments, the training module 800 may specifically be used for: Determine if user input has been received, and select the kernel function type. If so, then the kernel function of the Gaussian process regression model is determined based on the kernel function selection type; If not, the kernel function of the Gaussian process regression model is determined based on the preset kernel function type.

[0187] This invention also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the steps described in the various method embodiments above.

[0188] This embodiment also provides a chip, including a processor and a memory. The memory is used to store at least one instruction. When the instruction is loaded and executed by the processor, it implements the above-mentioned method for predicting welding quality or the training method for the welding quality prediction model. Its execution method and beneficial effects are similar and will not be described again here.

[0189] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0190] This is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting welding quality, characterized in that, include: Based on the post-weld image, the solder joints are extracted to form a solder joint image; Based on the solder joint diagram, determine the neural network features and image features corresponding to the post-soldering image; The neural network features and the image features are input into the welding quality prediction model to obtain the welding quality prediction value corresponding to the post-weld image.

2. The method according to claim 1, characterized in that, The step of determining the neural network features corresponding to the post-weld image based on the solder joint image includes: Based on the feature extraction model, the solder joint image is subjected to dimensionality reduction and feature extraction processing to obtain the neural network features corresponding to the solder joint image. Based on the neural network features corresponding to the solder joint image, the neural network features corresponding to the post-soldering image are determined.

3. The method according to claim 2, characterized in that, The training method for the feature extraction model includes: Based on historical post-weld images, historical solder joints are extracted to form a historical solder joint map; The historical solder joint image is projected onto the feature vector space to obtain feature values; The eigenvalues ​​in the eigenvector space are restored to obtain the reconstructed image; The difference between the reconstructed image and the historical solder joint image is determined, and the feature extraction model is trained based on the difference input model to form the feature extraction model.

4. The method according to claim 3, characterized in that, The step of performing dimensionality reduction and feature extraction on the solder joint image based on the feature extraction model to obtain the neural network features corresponding to the solder joint image includes: The solder joint image is projected onto the feature vector space of the feature extraction model to obtain the feature values ​​of the solder joint image; The feature values ​​of the solder joint image are determined as the neural network features corresponding to the solder joint image.

5. The method according to claim 1, characterized in that, The post-weld image has multiple solder joints. Determining the image features corresponding to the post-weld image based on the solder joint image includes: Based on the solder joint diagram, determine the total number of solder joints and the width and height of each solder joint; The diameter of a solder joint is determined based on the total number of solder joints and the width and height of each solder joint. The diameter of the weld point is determined as the image feature corresponding to the post-weld image.

6. The method according to claim 1, characterized in that, The step of determining the image features corresponding to the post-weld image based on the solder joint diagram includes: The grayscale value of each pixel in the solder joint image is fitted using a preset probability density function to obtain the scale parameter and shape parameter of the preset probability density function. The mode of the grayscale distribution of the solder joint image is determined based on the scale parameter of the preset probability density function and the shape parameter. The image features corresponding to the post-weld image are determined based on the mode of the grayscale distribution of the solder joint image.

7. The method according to claim 6, characterized in that, The method further includes: The grayscale standard deviation of the solder joint image is determined based on the scale parameter of the preset probability density function and the shape parameter. The image features corresponding to the post-weld image are determined based on the grayscale standard deviation of the solder joint image.

8. The method according to claim 1, characterized in that, The step of determining the image features corresponding to the post-weld image based on the solder joint diagram includes: The gray-level co-occurrence matrix of the solder joint image is determined based on the gray-level co-occurrence matrix algorithm. The texture complexity of the solder joint image is determined based on the gray-level co-occurrence matrix. The image features corresponding to the post-weld image are determined based on the texture complexity of the solder joint image.

9. The method according to any one of claims 1-8, characterized in that, The predicted welding quality value includes the predicted pull-out force value and the reliability of the predicted pull-out force value; the predicted welding quality model includes a Gaussian process regression model. The step of inputting the neural network features and the image features into the welding quality prediction model to obtain the welding quality prediction value corresponding to the post-weld image includes: The neural network features and the image features are standardized to obtain the input values; The mean and variance of the input values ​​are calculated using the kernel function of the Gaussian process regression model. The mean is determined as the predicted welding quality value corresponding to the post-weld image, and the confidence level of the predicted welding quality value is determined based on the variance.

10. A device for predicting welding quality, characterized in that, include: The solder joint extraction module is used to extract solder joints from the post-weld image and generate a solder joint map. The feature module is used to determine the neural network features and image features corresponding to the post-weld image based on the solder joint diagram; The prediction module is used to input the neural network features and the image features into the welding quality prediction model to obtain the welding quality prediction value corresponding to the post-weld image.

11. A training method for a welding quality prediction model, characterized in that, include: Based on the post-weld image, the solder joints are extracted to form a training solder joint image; Based on the trained solder joint diagram, neural network features and image features are determined; Based on the pull-out force test value corresponding to the training solder joint image, and combined with the neural network features and image features of the training solder joint image, a training sample is formed; A training sample set is constructed using the training samples corresponding to each of the training solder joint diagrams. The initial welding quality prediction model is trained using the training sample set, and the initial welding quality prediction model after the iteration ends is used as the welding quality prediction model.

12. The training method according to claim 11, characterized in that, The initial welding quality prediction model is a Gaussian process regression model; The step of training the initial welding quality prediction model using the training sample set includes: Standardize all training samples in the training sample set; Determine the kernel function of the Gaussian process regression model; Based on the standardized training samples, the hyperparameters of the kernel function are optimized by maximizing the marginal likelihood function to form the welding quality prediction model.

13. The training method according to claim 12, characterized in that, The process of determining the kernel function of the Gaussian process regression model includes: Determine if user input has been received, and select the kernel function type. If so, then the kernel function of the Gaussian process regression model is determined based on the kernel function selection type. If not, the kernel function of the Gaussian process regression model is determined based on the preset kernel function type.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as claimed in any one of claims 1-9 or 11-13.

15. A chip, characterized in that, include: A processor and a memory, the memory being used to store at least one instruction, which, when loaded and executed by the processor, implements the welding quality prediction method as described in any one of claims 1-9 or the training method for the welding quality prediction model as described in any one of claims 11-13.

Images