An intelligent visual detection method for welding fixtures

Abstract

The present invention discloses an intelligent visual detection method for welding fixtures, the method includes image acquisition, image enhancement, establishing a visual detection model for welding fixtures and visual detection of welding fixtures. The present invention belongs to the field of welding detection technology, specifically refers to an intelligent visual detection method for welding fixtures, the scheme extracts low-frequency components based on bilateral filtering to remove high-frequency noise in the image; constructs an adaptive optimization unit for color correction by designing Gaussian weights and activation functions; guides image correction by dynamically adjusting and defining upper and lower thresholds; thereby improving the accuracy of visual detection of welding fixtures; dynamically updates clustering based on source domain construction units to improve the generalization ability of the model; calculates the distance matrix and the loss of the source domain and target domain based on the source domain calculation unit; constructs a loss design unit to make the model optimization more comprehensive and accurate; and enables the visual detection model of welding fixtures to make a more stable and reliable final prediction.

Description

Technical Field

[0001] The invention relates to the technical field of welding detection, and in particular to an intelligent visualization detection method for welding fixtures. Background Art

[0002] The visual inspection method of welding fixtures refers to a method of automatically inspecting and analyzing welding fixtures using image processing and computer vision technology. This method helps to achieve real-time monitoring of fixture status, welding quality control, defect detection and maintenance management by processing and analyzing the image of the welding fixture. However, the general visual inspection method of welding fixtures has the problem that the original image is affected by a lot of noise, the edge information cannot be maintained when enhancing the image, the color correction is single, the enhancement effect is weak, and the recognition accuracy is poor; the general visual inspection method of welding fixtures has the problem that the model has weak generalization ability, insufficient feature extraction ability of welding fixture images, and single system loss setting, which leads to poor accuracy and stability of the model. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an intelligent visualization detection method for welding fixtures. In view of the problems that the original image of the general visualization detection method for welding fixtures is affected by a large amount of noise, the edge information cannot be maintained during image enhancement, the color correction is single, the enhancement effect is weak, and the recognition accuracy is poor, the scheme extracts low-frequency components based on bilateral filtering to remove high-frequency noise in the image; constructs adaptive optimization for color correction by designing Gaussian weights and activation functions; guides image correction by dynamically adjusting the upper and lower thresholds; introduces dense skip connections and cascade operations on the basis of standard U-Net to fuse features at different levels, improve the model's ability to retain details and enhance image quality; thereby making the visualization detection of welding fixtures more accurate. The measurement accuracy is improved; in view of the problems that the general welding fixture visualization detection method has weak model generalization ability, insufficient feature extraction ability for welding fixture images and single system loss setting, which lead to poor model accuracy and stability, this scheme dynamically updates the cluster center based on the source domain construction unit, and uses the network output and softmax function to generate pseudo labels for the target domain data to improve the generalization ability of the model; based on the source domain calculation unit, the distance matrix and the source domain and target domain losses are calculated to measure the performance of the model in different fields and optimize the model; the loss design unit is constructed by comprehensively considering the source domain sample loss, target domain sample loss, distance loss between cluster centers and consistency loss, so that the model optimization is more comprehensive and accurate; the welding fixture visualization detection model makes a more stable and reliable final prediction.

[0004] The technical solution adopted by the present invention is as follows: The present invention provides an intelligent welding fixture visual detection method, the method comprising the following steps:

[0005] Step S1: image acquisition;

[0006] Step S2: image enhancement;

[0007] Step S3: Establishing a visual inspection model for welding fixture;

[0008] Step S4: Visual inspection of welding fixture.

[0009] Furthermore, in step S1, the image acquisition is to acquire historical welding fixture images and real-time welding fixture images; and the historical welding fixture images are data labeled, and the expression type is welding quality inspection results.

[0010] Furthermore, in step S2, the image enhancement specifically includes the following steps:

[0011] Step S21: Image decomposition: obtaining the low-frequency component of the historical welding fixture image based on bilateral filtering, the formula used is as follows:

[0012] ;

[0013] In the formula, is the low-frequency component; BF(·) is the bilateral filtering operation; B is the input image;

[0014] Step S22: adaptive optimization; converting the low-frequency component into the HSI color space in advance; the adaptive optimization specifically includes:

[0015] Step S221: define a color correction function; the color correction function includes basic color differences, Gaussian weights of color guide images and an activation function; it is expressed as follows:

[0016] ;

[0017] Where E(I) is the energy representation of the color correction function; is the component in HSI color space; x and y are the pixel positions in the image; It is the reference value of basic color difference; is the value of the current image on the color component θ; is the adjustment parameter; G(·) is the Gaussian weight function; H(·) is the activation function; is the value of the saturation component; w(·) is the weight function; β is the parameter that controls the weight of the brightness component in color correction; is the value of the brightness component;

[0018] Step S222: define a Gaussian weight function; the Gaussian weight function is used to represent the adaptive adjustment ratio of the rule item to prevent pixel overflow; it is represented as follows:

[0019] ;

[0020] In the formula, is the image in color components The value on; t is the scale parameter;

[0021] Step S223: Design an activation function; the activation function is used to determine the time when the Gaussian weight starts to take effect; it is expressed as follows:

[0022] ;

[0023] ;

[0024] In the formula, is the color value vector of the image at position x; is a step function; is the image in color components The value on; c2 is the activation condition;

[0025] Step S224: Color-guided image correction; based on the image mean and mean square error, generate upper and lower thresholds for color-guided image correction; expressed as follows:

[0026] ;

[0027] In the formula, and are the upper and lower thresholds respectively; avg(·) is the mean; mse(·) is the mean square error; is a parameter that controls the weight of the mean square error in calculating the threshold;

[0028] Step S225: Color correct the image; the formula used is as follows:

[0029] ;

[0030] In the formula, is the color correction function;

[0031] Step S226: iterative optimization; constructing a criterion for terminating the iterative process; expressed as follows:

[0032] ;

[0033] In the formula, and They are and The image obtained in the iteration; is the iteration termination threshold;

[0034] Step S23: establishing an image enhancement model; taking the adaptively optimized historical welding fixture images as the original data set in advance, and dividing them into a quasi-training set and a quasi-testing set; establishing the image enhancement model specifically includes:

[0035] Step S231: model structure design; structural optimization is performed on the basis of the standard U-net. In the traditional U-net, the jump connection directly transfers the feature map of the encoder to the corresponding layer of the decoder; each pair of encoder and decoder layers are connected by a simple splicing operation; and the established image enhancement model not only connects adjacent layers, but also fuses the feature maps of multiple encoder layers to the decoder layer through dense jump connections; and in each jump connection, the image enhancement model adds a dense convolution module, and the dense convolution module in each jump connection is composed of multiple convolution layers; and a cascade operation is added to the jump connection, that is, the feature maps of the encoder and decoder are processed through multiple convolution layers and then connected; a feature fusion strategy is introduced, and in the decoder stage, the feature maps from different encoder layers are fused, and in the dense convolution module, local features are combined to enhance detail retention; and additional convolution layers and dense modules are introduced in the decoder;

[0036] Step S232: Design a loss function. The loss function is designed based on reconstruction loss, reflection component consistency loss and binary cross entropy, which is expressed as follows:

[0037] ;

[0038] Where L(·) is the total loss function; Y is the real image; is the enhanced image; is the weight parameter; N is the number of pixels; and are the real image value and model output image value of the b-th pixel respectively; is an estimate of the reflected component; is the preprocessed input image; is the target value of the reflected image; is a parameter used to balance various losses; It is used to control the reflection component and Parameters of consistency; is a parameter used to control the gradient loss; l and n are different layers of the image; is the gradient operator; I x is the image saturation; is a parameter used to control the gradient loss;

[0039] Step S233: image enhancement model determination; when the image enhancement model converges to the loss of the quasi-training set, the image enhancement model training is completed; based on the peak signal-to-noise ratio evaluation of the quasi-test set processed by the image enhancement model, an enhancement threshold is set in advance. If the peak signal-to-noise ratio of the quasi-test set before and after image enhancement is higher than the enhancement threshold, the image enhancement model is established; the original data set is processed based on the established image enhancement model and divided into a test set and a training set; otherwise, the model parameters are adjusted and the original data set is re-divided to train the image enhancement model.

[0040] Furthermore, in step S3, the establishment of a visual inspection model for welding fixtures specifically includes the following steps:

[0041] Step S31: data definition: defining source domain data instances and their labels, and target domain unlabeled data instances;

[0042] Step S32: feature extraction: extract low-level features, middle-level features and high-level features based on the pre-trained deep migration model ResNet, and perform global average pooling; expressed as follows:

[0043] ;

[0044] ;

[0045] Where F is the set of feature representations; l is the index of the feature layer, is a collection of feature layers; is the global average pooling operation; is the feature map; is the size; K(·) indicates whether to fuse the features of this layer; fusion means feature fusion; withoutfusion means no feature fusion;

[0046] Step S33: Source domain construction; used to calculate and update the cluster center of the source domain; for source domain data, the cluster center of the source domain is calculated through its label, and the cluster center is updated through the moving average rule; it is expressed as follows:

[0047] ;

[0048] In the formula, is the kth cluster center vector; and are the updated values ​​of the kth cluster center after the n1th iteration and the n1-1th iteration respectively; is the i3th eigenvector of the sample belonging to the kth class; α4 is the smoothing factor of the moving average;

[0049] For the target domain data, use the network output z and the softmax function Get class probabilities , and define pseudo labels ; is the i4th eigenvector of the sample belonging to the tth class;

[0050] Step S34: source domain calculation; used to calculate the distance matrix between the samples and the cluster centers of the source domain and the target domain, and calculate the loss; the cluster center is the central feature vector of each category in the source domain data; for the source domain data, the distance matrix between the samples and the cluster centers is constructed ; Establish the distance loss between the source domain samples and the cluster center , expressed as ; For the target domain data, calculate the distance matrix between the sample and the cluster center ; Establish the loss function through the entropy weighted matrix , expressed as: ;in, is the total number of samples in the source domain; is the sigmoid activation function; H(·) is the entropy function; is the total number of samples in the target domain;

[0051] Step S35: loss design; used to design a comprehensive loss function to optimize the model; construct a distance matrix between source domain cluster centers , the matrix elements are expressed as ; and calculate the cross entropy loss , expressed as: ; Then construct the joint loss function , expressed as: ; Establish cross entropy loss based on source domain data and its own labels , expressed as: ; For the cluster centers of the source domain and the target domain, construct an alignment loss function , expressed as: ; For cluster centers and network neurons, construct consistency loss , expressed as: ; Final construction welding fixture quality inspection loss , expressed as: ; Where c is the number of cluster centers; is the softmax activation function; and are the cross entropy losses of the distance matrices between the source domain cluster centers belonging to the s-th class samples and the t-th class samples; is the feature vector of the source domain sample; and are the kth cluster centers of the source domain and the target domain respectively; , and They are all weight coefficients of loss terms; It is the output of the kth cluster center through the last layer of the network;

[0052] Step S36: Final prediction; for each category c1, calculate the sum of all classification probabilities on the category, and find the category with the maximum probability and the corresponding category; if the predicted welding quality test result is unqualified, visualize the image at the welding interface; the final prediction is expressed as follows:

[0053] ;

[0054] In the formula, is the final predicted category label; is the probability value of the i3th sample in the c1th category.

[0055] Further, in step S4, the welding fixture visualization detection is based on the established welding fixture visualization detection model, the real-time welding fixture image is input into the welding fixture visualization detection model, and the welding fixture visualization detection result is realized based on the welding fixture visualization detection output.

[0056] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0057] (1) Aiming at the problems that the original image of the general welding fixture visualization detection method is affected by a lot of noise, the edge information cannot be maintained during image enhancement, the color correction is monotonous, the enhancement effect is weak, and the recognition accuracy is poor, this scheme extracts low-frequency components based on bilateral filtering to remove high-frequency noise in the image; constructs adaptive optimization for color correction by designing Gaussian weights and activation functions; guides image correction by dynamically adjusting the upper and lower thresholds; introduces dense skip connections and cascade operations on the basis of standard U-Net to fuse features at different levels, improve the model's ability to retain details and enhance image quality; thereby improving the accuracy of welding fixture visualization detection.

[0058] (2) In view of the problems of weak generalization ability of the model, insufficient feature extraction ability of welding fixture images and single system loss setting in general welding fixture visualization detection methods, which lead to poor accuracy and stability of the model, this scheme dynamically updates the cluster centers based on the source domain construction unit, and uses the network output and softmax function to generate pseudo labels for the target domain data to improve the generalization ability of the model; based on the source domain calculation unit, the distance matrix and the source domain and target domain losses are calculated to measure the performance of the model in different fields and optimize the model; the loss design unit is constructed by comprehensively considering the source domain sample loss, target domain sample loss, distance loss between cluster centers and consistency loss, so as to make the model optimization more comprehensive and accurate; and the welding fixture visualization detection model can make more stable and reliable final predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a flow chart of an intelligent welding fixture visualization detection method provided by the present invention;

[0060] Figure 2 is a schematic flow chart of step S2;

[0061] Figure 3 It is a flowchart diagram of step S22.

[0062] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0064] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0065] Example 1, see Figure 1 The present invention provides an intelligent welding fixture visual detection method, which includes the following steps:

[0066] Step S1: image acquisition, acquiring historical welding fixture images and real-time welding fixture images;

[0067] Step S2: Image enhancement, extract low-frequency components based on bilateral filtering to remove high-frequency noise in the image; construct adaptive optimization by designing Gaussian weights and activation functions to perform color correction; define upper and lower thresholds through dynamic adjustment to guide image correction; introduce dense skip connections and cascade operations based on the standard U-Net; and perform enhancement operations on the welding fixture image;

[0068] Step S3: Establish a visual detection model for welding fixtures, dynamically update the cluster centers based on the source domain construction unit, and use the network output and the softmax function to generate pseudo labels for the target domain data; calculate the distance matrix and the source domain and target domain losses based on the source domain calculation unit; comprehensively consider the source domain sample loss, target domain sample loss, distance loss between cluster centers, and consistency loss to construct a loss design unit, and then complete the establishment of the visual detection model for welding fixtures;

[0069] Step S4: Visual inspection of welding fixture.

[0070] Example 2, see Figure 1 This embodiment is based on the above embodiment. In step S1, data annotation is performed on the historical welding fixture image, and the expression type is the welding quality inspection result.

[0071] Example 3, see Figure 1 , Figure 2 and Figure 3 This embodiment is based on the above embodiment. In step S2, image enhancement specifically includes the following steps:

[0072] Step S21: Image decomposition: obtaining the low-frequency component of the historical welding fixture image based on bilateral filtering, the formula used is as follows:

[0073] ;

[0074] In the formula, is the low-frequency component; BF(·) is the bilateral filtering operation; B is the input image;

[0075] Step S22: adaptive optimization; pre-converting the low-frequency component to the HSI color space; the adaptive optimization specifically includes:

[0076] Step S221: define a color correction function; the color correction function includes basic color differences, Gaussian weights of color guide images and an activation function; it is expressed as follows:

[0077] ;

[0078] Where E(I) is the energy representation of the color correction function; is the component in HSI color space; x and y are the pixel positions in the image; It is the reference value of basic color difference; is the value of the current image on the color component θ; is the adjustment parameter; G(·) is the Gaussian weight function; H(·) is the activation function; is the value of the saturation component; w(·) is the weight function; β is the parameter that controls the weight of the brightness component in color correction; is the value of the brightness component;

[0079] Step S222: define a Gaussian weight function; the Gaussian weight function is used to represent the adaptive adjustment ratio of the rule item to prevent pixel overflow; it is represented as follows:

[0080] ;

[0081] In the formula, is the image in color components The value on; t is the scale parameter;

[0082] Step S223: Design an activation function; the activation function is used to determine the time when the Gaussian weight starts to take effect; it is expressed as follows:

[0083] ;

[0084] ;

[0085] In the formula, is the color value vector of the image at position x; is a step function; is the image in color components The value on; c2 is the activation condition;

[0086] Step S224: Color-guided image correction; based on the image mean and mean square error, generate upper and lower thresholds for color-guided image correction; expressed as follows:

[0087] ;

[0088] In the formula, and are the upper and lower thresholds respectively; avg(·) is the mean; mse(·) is the mean square error; is a parameter that controls the weight of the mean square error in calculating the threshold;

[0089] Step S225: Color correct the image; the formula used is as follows:

[0090] ;

[0091] In the formula, is the color correction function;

[0092] Step S226: iterative optimization; constructing a criterion for terminating the iterative process; expressed as follows:

[0093] ;

[0094] In the formula, and They are and The image obtained in the iteration; is the iteration termination threshold;

[0095] Step S23: establishing an image enhancement model; taking the adaptively optimized historical welding fixture images as the original data set in advance, and dividing them into a quasi-training set and a quasi-testing set; establishing the image enhancement model specifically includes:

[0096] Step S231: Model structure design; structural optimization is performed on the basis of the standard U-net. In the traditional U-net, the jump connection directly transfers the feature map of the encoder to the corresponding layer of the decoder; each pair of encoder and decoder layers are connected by a simple splicing operation; and the established image enhancement model not only connects adjacent layers, but also fuses the feature maps of multiple encoder layers to the decoder layer through dense jump connections; and in each jump connection, the image enhancement model adds a dense convolution module, and the dense convolution module in each jump connection is composed of multiple convolution layers; and a cascade operation is added to the jump connection, that is, the feature maps of the encoder and decoder are processed through multiple convolution layers and then connected; a feature fusion strategy is introduced, and in the decoder stage, the feature maps from different encoder layers are fused, and in the dense convolution module, local features are combined to enhance detail retention; and additional convolution layers and dense modules are introduced in the decoder;

[0097] Step S232: Design a loss function. The loss function is designed based on reconstruction loss, reflection component consistency loss and binary cross entropy, which is expressed as follows:

[0098] ;

[0099] Where L(·) is the total loss function; Y is the real image; is the enhanced image; is the weight parameter; N is the number of pixels; and are the real image value and model output image value of the b-th pixel respectively; is an estimate of the reflected component; is the preprocessed input image; is the target value of the reflected image; is a parameter used to balance various losses; It is used to control the reflection component and Parameters of consistency; is a parameter used to control the gradient loss; l and n are different layers of the image; is the gradient operator; I x is the image saturation; is a parameter used to control the gradient loss;

[0100] Step S233: image enhancement model determination; when the image enhancement model converges to the loss of the quasi-training set, the image enhancement model training is completed; based on the peak signal-to-noise ratio evaluation of the quasi-test set processed by the image enhancement model, an enhancement threshold is set in advance. If the peak signal-to-noise ratio of the quasi-test set before and after image enhancement is higher than the enhancement threshold, the image enhancement model is established; the original data set is processed based on the established image enhancement model and divided into a test set and a training set; otherwise, the model parameters are adjusted and the original data set is re-divided to train the image enhancement model.

[0101] By performing the above operations, the general welding fixture visualization detection method has a lot of noise influence on the original image, the edge information cannot be maintained during image enhancement, the color correction is simplistic, the enhancement effect is weak, and the recognition accuracy is poor. This scheme extracts low-frequency components based on bilateral filtering to remove high-frequency noise in the image; constructs adaptive optimization for color correction by designing Gaussian weights and activation functions; guides image correction by dynamically adjusting the upper and lower thresholds; introduces dense skip connections and cascade operations on the basis of standard U-Net to fuse features at different levels, improve the model's ability to retain details and enhance image quality; and improves the accuracy of welding fixture visualization detection. The global search capability of the method; introduces T-distribution to design moving weights to improve the accuracy and robustness of the algorithm in the optimization process.

[0102] Example 4, see Figure 1 This embodiment is based on the above embodiment. In step S3, establishing a visual detection model for welding fixtures specifically includes the following steps:

[0103] Step S31: data definition: defining source domain data instances and their labels, and target domain unlabeled data instances;

[0104] Step S32: feature extraction: extract low-level features, middle-level features and high-level features based on the pre-trained deep migration model ResNet, and perform global average pooling; expressed as follows:

[0105] ;

[0106] ;

[0107] Where F is the set of feature representations; l is the index of the feature layer, is a collection of feature layers; is the global average pooling operation; is the feature map; is the size; K(·) indicates whether to fuse the features of this layer; fusion means feature fusion; withoutfusion means no feature fusion;

[0108] Step S33: Source domain construction; used to calculate and update the cluster center of the source domain; for source domain data, the cluster center of the source domain is calculated through its label, and the cluster center is updated through the moving average rule; it is expressed as follows:

[0109] ;

[0110] In the formula, is the kth cluster center vector; and are the updated values ​​of the kth cluster center after the n1th iteration and the n1-1th iteration respectively; is the i3th eigenvector of the sample belonging to the kth class; α4 is the smoothing factor of the moving average;

[0111] For the target domain data, use the network output z and the softmax function Get class probabilities , and define pseudo labels ; is the i4th eigenvector of the sample belonging to the tth class;

[0112] Step S34: source domain calculation; used to calculate the distance matrix between the samples and the cluster centers of the source domain and the target domain, and calculate the loss; the cluster center is the central feature vector of each category in the source domain data; for the source domain data, the distance matrix between the samples and the cluster centers is constructed ; Establish the distance loss between the source domain samples and the cluster center , expressed as ; For the target domain data, calculate the distance matrix between the sample and the cluster center ; Establish the loss function through the entropy weighted matrix , expressed as: ;in, is the total number of samples in the source domain; is the sigmoid activation function; H(·) is the entropy function; is the total number of samples in the target domain;

[0113] Step S35: loss design; used to design a comprehensive loss function to optimize the model; construct a distance matrix between source domain cluster centers , the matrix elements are expressed as ; and calculate the cross entropy loss , expressed as: ; Then construct the joint loss function , expressed as: ; Establish cross entropy loss based on source domain data and its own labels , expressed as: ; For the cluster centers of the source domain and the target domain, construct an alignment loss function , expressed as: ; For cluster centers and network neurons, construct consistency loss , expressed as: ; Final construction welding fixture quality inspection loss , expressed as: ; Where c is the number of cluster centers; is the softmax activation function; and are the cross entropy losses of the distance matrices between the source domain cluster centers belonging to the s-th class samples and the t-th class samples; is the feature vector of the source domain sample; and are the kth cluster centers of the source domain and the target domain respectively; , and They are all weight coefficients of loss terms; It is the output of the kth cluster center through the last layer of the network;

[0114] Step S36: Final prediction; for each category c1, calculate the sum of all classification probabilities on the category, and find the category with the maximum probability and the corresponding category; if the predicted welding quality test result is unqualified, visualize the image at the welding interface; the final prediction is expressed as follows:

[0115] ;

[0116] In the formula, is the final predicted category label; is the probability value of the i3th sample in the c1th category.

[0117] By performing the above operations, in view of the problems of weak generalization ability of the model, insufficient feature extraction ability of welding fixture images and single system loss setting in general welding fixture visualization detection methods, which lead to poor accuracy and stability of the model, this scheme dynamically updates the cluster centers based on the source domain construction unit, and uses the network output and softmax function to generate pseudo labels for the target domain data to improve the generalization ability of the model; calculates the distance matrix and the source domain and target domain losses based on the source domain calculation unit, measures the performance of the model in different fields, and optimizes the model; comprehensively considers the source domain sample loss, target domain sample loss, distance loss between cluster centers and consistency loss to construct the loss design unit, so that the model optimization is more comprehensive and accurate; and enables the welding fixture visualization detection model to make a more stable and reliable final prediction.

[0118] Example 5, see Figure 1 This embodiment is based on the above embodiment. In step S4, the welding fixture visualization detection is based on the established welding fixture visualization detection model. The real-time welding fixture image is input into the welding fixture visualization detection model, and the welding fixture visualization detection result is realized based on the welding fixture visualization detection output.

[0119] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0120] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0121] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. An intelligent visual detection method for welding fixtures, characterized in that: The method comprises the following steps: Step S1: image acquisition; Step S2: Image enhancement, extract low-frequency components based on bilateral filtering to remove high-frequency noise in the image; construct adaptive optimization by designing Gaussian weights and activation functions to perform color correction; define upper and lower thresholds through dynamic adjustment to guide image correction; introduce dense skip connections and cascade operations based on the standard U-Net; and perform enhancement operations on the welding fixture image; Step S3: Establish a visual detection model for welding fixtures, dynamically update the cluster centers based on the source domain construction unit, and use the network output and the softmax function to generate pseudo labels for the target domain data; calculate the distance matrix and the source domain and target domain losses based on the source domain calculation unit; comprehensively consider the source domain sample loss, target domain sample loss, distance loss between cluster centers, and consistency loss to construct a loss design unit, and then complete the establishment of the visual detection model for welding fixtures; Step S4: Visual inspection of welding fixture.

2. According to claim 1, the intelligent visual detection method for welding fixtures is characterized by: In step S2, the image enhancement specifically includes the following steps: Step S21: Image decomposition: obtaining the low-frequency component of the historical welding fixture image based on bilateral filtering, the formula used is as follows: ； In the formula, is the low-frequency component; BF(·) is the bilateral filtering operation; B is the input image; Step S22: adaptive optimization; converting the low-frequency component into the HSI color space in advance; the adaptive optimization specifically includes: Step S221: define a color correction function; the color correction function includes basic color differences, Gaussian weights of color guide images and an activation function; it is expressed as follows: ； Where E(I) is the energy representation of the color correction function; is the component in HSI color space; x and y are the pixel positions in the image; It is the reference value of basic color difference; is the value of the current image on the color component θ; is the adjustment parameter; G(·) is the Gaussian weight function; H(·) is the activation function; is the value of the saturation component; w(·) is the weight function; β is the parameter that controls the weight of the brightness component in color correction; is the value of the brightness component; Step S222: define a Gaussian weight function; the Gaussian weight function is used to represent the adaptive adjustment ratio of the rule item to prevent pixel overflow; it is represented as follows: ； In the formula, is the image in color components The value on; t is the scale parameter; Step S223: Design an activation function; the activation function is used to determine the time when the Gaussian weight starts to take effect; it is expressed as follows: ； ； In the formula, is the color value vector of the image at position x; is a step function; is the image in color components The value on; c2 is the activation condition; Step S224: Color-guided image correction; based on the image mean and mean square error, generate upper and lower thresholds for color-guided image correction; expressed as follows: ； In the formula, and are the upper and lower thresholds respectively; avg(·) is the mean; mse(·) is the mean square error; is a parameter that controls the weight of the mean square error in calculating the threshold; Step S225: Color correct the image; the formula used is as follows: ； In the formula, is the color correction function; Step S226: iterative optimization; constructing a criterion for terminating the iterative process; expressed as follows: ； In the formula, and They are and The image obtained in the iteration; is the iteration termination threshold; Step S23: establishing an image enhancement model; taking the adaptively optimized historical welding fixture images as the original data set in advance, and dividing them into a quasi-training set and a quasi-testing set; establishing the image enhancement model specifically includes: Step S231: Model structure design; structural optimization is performed on the basis of the standard U-net. In the traditional U-net, the jump connection directly transfers the feature map of the encoder to the corresponding layer of the decoder; each pair of encoder and decoder layers are connected by a simple splicing operation; and the established image enhancement model not only connects adjacent layers, but also fuses the feature maps of multiple encoder layers to the decoder layer through dense jump connections; and in each jump connection, the image enhancement model adds a dense convolution module, and the dense convolution module in each jump connection is composed of multiple convolution layers; and a cascade operation is added to the jump connection, that is, the feature maps of the encoder and decoder are processed through multiple convolution layers and then connected; a feature fusion strategy is introduced, and in the decoder stage, the feature maps from different encoder layers are fused, and in the dense convolution module, local features are combined to enhance detail retention; and additional convolution layers and dense modules are introduced in the decoder; Step S232: Design a loss function. The loss function is designed based on reconstruction loss, reflection component consistency loss and binary cross entropy, which is expressed as follows: ； Where L(·) is the total loss function; Y is the real image; is the enhanced image; is the weight parameter; N is the number of pixels; and are the real image value and model output image value of the b-th pixel respectively; is an estimate of the reflected component; is the preprocessed input image; is the target value of the reflected image; is a parameter used to balance various losses; It is used to control the reflection component and Parameters of consistency; is a parameter used to control the gradient loss; l and n are different layers of the image; is the gradient operator; I x is the image saturation; is a parameter used to control the gradient loss; Step S233: image enhancement model determination; when the image enhancement model converges to the loss of the quasi-training set, the image enhancement model training is completed; based on the peak signal-to-noise ratio evaluation of the quasi-test set processed by the image enhancement model, an enhancement threshold is set in advance. If the peak signal-to-noise ratio of the quasi-test set before and after image enhancement is higher than the enhancement threshold, the image enhancement model is established; the original data set is processed based on the established image enhancement model and divided into a test set and a training set; otherwise, the model parameters are adjusted and the original data set is re-divided to train the image enhancement model.

3. The intelligent visual detection method for welding fixture according to claim 1 is characterized in that: In step S3, the establishment of a visual inspection model for welding fixtures specifically includes the following steps: Step S31: data definition: defining source domain data instances and their labels, and target domain unlabeled data instances; Step S32: feature extraction: extract low-level features, middle-level features and high-level features based on the pre-trained deep migration model ResNet, and perform global average pooling; expressed as follows: ； ； Where F is the set of feature representations; l is the index of the feature layer, is a collection of feature layers; is the global average pooling operation; is the feature map; is the size; K(·) indicates whether to fuse the features of this layer; fusion means feature fusion; withoutfusion means no feature fusion; Step S33: Source domain construction; used to calculate and update the cluster center of the source domain; for source domain data, the cluster center of the source domain is calculated through its label, and the cluster center is updated through the moving average rule; it is expressed as follows: ； In the formula, is the kth cluster center vector; and are the updated values ​​of the kth cluster center after the n1th iteration and the n1-1th iteration respectively; is the i3th eigenvector of the sample belonging to the kth class; α4 is the smoothing factor of the moving average; For the target domain data, use the network output z and the softmax function Get class probabilities , and define pseudo labels ; is the i4th eigenvector of the sample belonging to the tth class; Step S34: source domain calculation; used to calculate the distance matrix between the samples and the cluster centers of the source domain and the target domain, and calculate the loss; the cluster center is the central feature vector of each category in the source domain data; for the source domain data, the distance matrix between the samples and the cluster centers is constructed ; Establish the distance loss between the source domain samples and the cluster center , expressed as ; For the target domain data, calculate the distance matrix between the sample and the cluster center ; Establish the loss function through the entropy weighted matrix , expressed as: ;in, is the total number of samples in the source domain; is the sigmoid activation function; H(·) is the entropy function; is the total number of samples in the target domain; Step S35: loss design; used to design a comprehensive loss function to optimize the model; construct a distance matrix between source domain cluster centers , the matrix elements are expressed as ; and calculate the cross entropy loss , expressed as: ; Then construct the joint loss function , expressed as: ; Establish cross entropy loss based on source domain data and its own labels , expressed as: ; For the cluster centers of the source domain and the target domain, construct an alignment loss function , expressed as: ; For cluster centers and network neurons, construct consistency loss , expressed as: ; Final construction welding fixture quality inspection loss , expressed as: ; Where c is the number of cluster centers; is the softmax activation function; and are the cross entropy losses of the distance matrices between the source domain cluster centers belonging to the s-th class samples and the t-th class samples; is the feature vector of the source domain sample; and are the kth cluster centers of the source domain and the target domain respectively; , and They are all weight coefficients of loss terms; It is the output of the kth cluster center through the last layer of the network; Step S36: Final prediction; for each category c1, calculate the sum of all classification probabilities on the category, and find the category with the maximum probability and the corresponding category; if the predicted welding quality test result is unqualified, visualize the image at the welding interface; the final prediction is expressed as follows: ； In the formula, is the final predicted category label; is the probability value of the i3th sample in the c1th category.

4. The intelligent visual detection method for welding fixture according to claim 1 is characterized in that: In step S1, the image acquisition is to acquire historical welding fixture images and real-time welding fixture images; and the historical welding fixture images are data labeled, and the expression type is welding quality inspection results.

5. The intelligent visual detection method for welding fixture according to claim 1 is characterized in that: In step S4, the welding fixture visual detection is based on the established welding fixture visual detection model, the real-time welding fixture image is input into the welding fixture visual detection model, and the welding fixture visual detection result is realized based on the welding fixture visual detection output.

Images