Weld quality inspection method based on machine vision

Abstract

The present application relates to the technical field of quality inspection, and in particular to a weld quality inspection method based on machine vision. The method comprises the following steps: obtaining a surface image of a weld by a visible light camera, and dividing the surface image of the weld into multiple segments; generating a thermal excitation parameter time series change sequence of each segment by a preset heating parameter time series generation model according to the surface image of each segment; performing thermal excitation heating on the weld area of ​​each segment according to the thermal excitation parameter time series change sequence of each segment; in the process of thermal excitation heating, obtaining multiple infrared images of each segment along the time series by an infrared camera to obtain a time series change sequence of infrared images of each segment; and inspecting the weld quality of each segment by a preset weld quality inspection model according to the time series change sequence of infrared images of each segment. The weld quality inspection method based on machine vision provided by the present application can improve the accuracy of weld quality inspection.

Description

Technical Field

[0001] The present application relates to the technical field of quality inspection, and in particular to a weld quality inspection method based on machine vision. Background Art

[0002] With the continuous development of industrial manufacturing, welding, as a key connection process, has been widely used in machinery manufacturing, automobile manufacturing, aerospace and other fields. The quality of welds directly affects the performance and safety of products. Therefore, efficient and accurate detection of weld quality is of great significance.

[0003] The existing infrared photography detection method uses a thermal excitation device to heat the weld to be tested, so that the weld generates heat after being heated. For uniform parts of the weld, the heat is evenly distributed, while in non-uniform parts with defects, the heat generated will be different due to the inconsistency of material properties and structure. In the weld quality inspection based on infrared photography, in order to capture the changing temperature characteristics, the equipment and model need to have high sensitivity, and the detection cost and complexity are high.

[0004] Therefore, the existing technology still has shortcomings in the accuracy of weld quality detection and needs to be improved urgently. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a weld quality inspection method based on machine vision, which can improve the accuracy of weld quality inspection.

[0006] The present application provides a weld quality detection method based on machine vision, comprising the following steps:

[0007] Acquire a surface image of the weld by using a visible light camera, and divide the surface image of the weld into a plurality of segments;

[0008] According to the surface image of each segment, a time series variation sequence of thermal excitation parameters of each segment is generated by a preset heating parameter time series generation model;

[0009] Thermally excitation heating is performed on the weld area of ​​each segment according to the time series variation sequence of thermal excitation parameters of each segment;

[0010] During the thermal excitation heating process, a plurality of infrared images along the time sequence of each segment are acquired by an infrared camera to obtain a time sequence change sequence of infrared images of each segment;

[0011] The weld quality of each segment is inspected according to the time series change sequence of the infrared images of each segment through a preset weld quality inspection model.

[0012] Optionally, the weld quality detection method based on machine vision further includes the following steps:

[0013] A weld heat treatment step is provided before the surface image of the weld is acquired by the camera;

[0014] After the weld is heat treated and before the surface image of the weld is acquired by the visible light camera, an infrared pre-acquisition image of each segment is acquired by the infrared camera;

[0015] Furthermore, according to the infrared pre-acquisition image of each segment, a thermal excitation parameter time series variation sequence of each segment is generated through a preset heating parameter time series generation model.

[0016] Optionally, dividing the surface image of the weld into a plurality of segments comprises the following steps:

[0017] Acquire a temperature abnormality area in the infrared pre-acquisition image whose temperature is abnormal compared with other areas through the infrared pre-acquisition image;

[0018] Taking the center point of the temperature anomaly area as the center seed point, a surface image segment of a preset size is divided;

[0019] The following steps are repeated until the surface image of the weld is completely divided into multiple segments:

[0020] A central seed point is randomly set in the undivided remaining area of ​​the surface image, and the division is continued towards the undivided remaining area until the shape size defined by each segment is reached.

[0021] Optionally, the heating parameter time series generation model is trained by the following steps:

[0022] Constructing the heating parameter timing generation model, the heating parameter timing generation model includes a timing generation model input layer, a feature extraction layer, a feature decoupler layer, a timing encoder layer, a timing attention mechanism fusion layer, a heating parameter sequence generation module layer and a timing generation model output layer;

[0023] The input layer of the temporal generation model is used to receive the infrared pre-acquisition image and the surface image of each segment;

[0024] The feature extraction layer is connected to the temporal generation model input layer to obtain the infrared pre-acquisition image and the surface image of each segment. The feature extraction layer includes an infrared pre-acquisition image feature extractor and a surface defect feature extractor. The infrared pre-acquisition image feature extractor is a first deep convolution module. The first deep convolution module is composed of multiple convolution modules and pooling modules to obtain the feature representation F of the infrared pre-acquisition image through multiple layers of convolution. heat The surface defect feature extractor is a second deep convolution module, which is composed of multiple convolution modules and pooling modules to obtain the surface feature representation F of the surface image through multi-layer convolution. surface ;

[0025] The feature decoupler layer is connected to the feature extraction layer to obtain the feature representation F of the infrared pre-acquisition image. heat and surface characteristics F surface The feature decoupler layer includes a feature decoupler input layer, a decoupled convolutional neural network, and a feature decoupler output layer. The feature decoupler input layer obtains the surface feature representation F surface And input it into the decoupled convolutional neural network to obtain the feature representation F of the infrared pre-acquired image heat Dimensionally consistent surface defects affect the representation of features F surface-heat , the feature decoupler output layer obtains the feature representation F of the infrared pre-acquisition image heat And the feature is represented by F heat Subtract surface defects affecting feature F surface-heat In this way, we get the internal feature representation F internal ;

[0026] The temporal encoder layer encodes each time step according to the preset total heating time T and the preset time step t to obtain the time encoding vector E t ;

[0027] The temporal attention mechanism fusion layer includes a weight calculation module and a feature fusion module. The weight calculation module is a first multi-layer perceptron. The first multi-layer perceptron is used to calculate the time encoding vector E according to the time encoding vector E. t Generate the surface feature weight α corresponding to each time step t and the internal feature weight β t ;

[0028] The feature fusion module is used to calculate the surface feature weight α t and the internal feature weight β t The surface feature F surface and the internal feature representation F internal Fusion is performed to obtain the fusion feature F of time step t fused，t ;

[0029] A heating parameter sequence generation module layer, wherein the heating parameter sequence generation module layer comprises a sequence input layer and a first Transformer decoder, wherein the sequence input layer is used to receive a fusion feature sequence (F fused，1 , F fused，2 , F fused，3 , ..., F fused，T ) and the sequence of time-encoded vectors (E1, E2, E3, …, E T ), the first Transformer decoder receives the fused feature sequence and the time encoding vector sequence through the sequence input layer to generate a thermal excitation parameter temporal change sequence;

[0030] A time series generation model output layer, wherein the time series generation model output layer receives the output of the heating parameter sequence generation module layer to output a thermal excitation parameter time series change sequence to the outside;

[0031] A plurality of thermal excitation parameter time series change sequences and corresponding surface images and infrared pre-acquisition images are collected as training sets, and the surface images and infrared pre-acquisition images are used as inputs of the heating parameter time series generation model, and the corresponding thermal excitation parameter time series change sequences are used as training labels to train and obtain the heating parameter time series generation model.

[0032] Optionally, the decoupled convolutional neural network is obtained by the following steps:

[0033] Collecting a plurality of surface images with only surface defects and corresponding infrared pre-acquisition images;

[0034] The surface image is processed into the feature representation F of the infrared pre-acquisition image through the same model structure as the second deep convolution module heat ;

[0035] The infrared pre-acquired image is processed into a surface feature representation F through the same model structure as the first deep convolution module. surface ;

[0036] The convolutional neural network is used as the network framework of the decoupled convolutional neural network, and the surface features are used to represent F surface As the input of the convolutional neural network, the feature representation F of the infrared pre-acquired image heat As the label, the first loss function is used as the loss function, and the decoupled convolutional neural network is trained in this way;

[0037] The first loss function is:

[0038]

[0039] Among them, surface defects affect the characteristic F surface-heat is the prediction output of the decoupled convolutional neural network.

[0040] Optionally, the weld quality detection model is trained by the following steps:

[0041] Long short-term memory network is used as the network framework of weld quality detection model;

[0042] Multiple infrared image time series change sequences and the corresponding quality scores are collected as the training set of the weld quality detection model. The infrared image time series change sequence is used as the input of the long short-term memory network, and the corresponding quality score is used as the label of the long short-term memory network to train the weld quality detection model.

[0043] Compared with the prior art, the technical solution provided by this application has the following advantages:

[0044] One of the beneficial effects is that the present application can use a visible light camera to obtain a surface image of the weld, and for each segment of the surface image, a preset heating parameter timing generation model is used to generate a corresponding thermal excitation parameter timing change sequence. Specifically, this is because different surface defects will affect the heat conduction path inside the weld. By identifying the location, shape and type of surface defects through machine vision, the model can adjust the heating parameters, such as heating power, frequency and duration, based on these features. This targeted heating can produce a more significant temperature difference between the defective area and the surrounding normal area, enhancing the thermal contrast. Moreover, when surface defects and internal defects exist at the same time, the surface defects are generally associated with the internal defects. Through this correlation feature between surface defects and internal defects, the model can also enhance the thermal response of internal defects.

[0045] Subsequently, the present application can perform thermal excitation heating on the weld area of ​​each segment according to the generated thermal excitation parameter time series change sequence. In this process, multiple infrared images along the time series of each segment are acquired by an infrared camera to form a time series change sequence of infrared images. The infrared images contained in the sequence not only enhance the thermal response of known defects, but also capture the thermal characteristics of other possible internal defects due to the improved overall thermal contrast caused by the optimized heating process.

[0046] Finally, based on the temporal change sequence of infrared images in each segment, the preset weld quality inspection model is used to conduct a comprehensive inspection of the weld quality. This model can accurately identify quality problems in welds by analyzing the temporal and spatial characteristics of infrared images.

[0047] Therefore, the weld quality inspection method based on machine vision provided in this application can improve the accuracy of weld quality inspection.

[0048] The second beneficial effect is that stress elimination by heat treatment of welds is a common processing step. The present application achieves two effects at the same time by using an infrared camera to obtain an infrared pre-acquisition image of each segment after heat treatment and before obtaining a surface image. The first effect is that the temperature field characteristics of the weld after heat treatment can be used to quickly identify abnormal temperature areas on the surface and inside of the weld. Through these infrared pre-acquisition images, the complete position of surface defects or internal defects can be quickly located, avoiding the occurrence of segment division errors in the subsequent image segmentation process. This pre-scanning method effectively reduces the detection error caused by division errors and improves the accuracy of detection.

[0049] The second effect is that since both the surface defects and internal defects of the weld will have different effects on the thermal conduction and thermal radiation characteristics, when the surface defects and internal defects are superimposed, the heating curve generated directly based on the surface image does not take into account the thermal response of the internal defects, which affects the detection results. In addition, excessive reliance on surface features to generate heating curves will lead to overfitting of the model, that is, the generated thermal excitation parameter time series will be more prominent in expressing the thermal response caused by surface defects, resulting in internal defects being concealed to a certain extent.

[0050] By acquiring an infrared pre-acquisition image before the complete weld inspection, the present application can obtain the temperature distribution information of the weld before the formal inspection, thereby introducing part of the internal defect information and assisting in determining the best heating parameter combination. Therefore, the weld quality inspection method based on machine vision provided by the present application can improve the accuracy of weld quality inspection.

[0051] The third beneficial effect is that the weld quality detection method based on machine vision provided by the present application introduces a feature decoupling mechanism when constructing a heating parameter timing generation model, and separates the superimposed thermal features into independent surface features and internal features. Through the feature decoupler layer, a decoupled convolutional neural network is used to remove the influence of surface defects from the feature representation of the infrared pre-acquired image to obtain a pure internal feature representation. This process ensures that the features of surface defects and internal defects can be identified and analyzed separately, avoiding interference between each other and improving the accuracy of defect detection.

[0052] On this basis, the present application further combines a temporal attention mechanism module to dynamically adjust the attention to surface features and internal features according to different time series during the heating process. In the early stage of heating, the heat is mainly concentrated in the surface area of ​​the material, and the thermal response and thermal contrast of surface defects are more obvious. Therefore, the temporal attention mechanism gives higher weight to surface features at this stage. As the heating progresses, the heat gradually conducts into the interior of the material, the thermal response of internal defects begins to increase, and the temporal attention mechanism gradually increases the weight of internal features.

[0053] This dynamic adjustment strategy enables the system to generate more appropriate heating parameters at different heating stages and specifically enhance the thermal contrast of different defects. Through the heating parameter sequence generation module layer, according to the fusion features along the time series output by the time series attention mechanism fusion layer, the model can generate a time series change sequence of thermal excitation parameters that meets actual needs. This sequence is optimized at each time step to ensure effective detection of surface and internal defects.

[0054] Therefore, the weld quality inspection method based on machine vision provided in this application can improve the accuracy of weld quality inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of the structure of the heating parameter timing generation model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application can also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, not all of the embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0058] The present application provides a method for detecting weld quality based on machine vision, comprising the following steps:

[0059] S100: a weld heat treatment step is provided before the surface image of the weld is acquired by the camera. After the weld heat treatment and before the surface image of the weld is acquired by the visible light camera, an infrared pre-acquisition image of the weld is acquired by the infrared camera.

[0060] The weld heat treatment step is a conventional step of heating the weld to relieve stress after welding is completed.

[0061] After heat treatment, the infrared pre-acquisition image of the entire weld can be completely obtained by the infrared camera.

[0062] S101: Acquire a surface image of a weld by using a visible light camera, and divide the surface image of the weld and an infrared pre-acquired image into a plurality of segments.

[0063] Specifically, dividing the surface image of the weld into multiple segments comprises the following steps:

[0064] Acquire a temperature abnormality area in the infrared pre-acquisition image whose temperature is abnormal compared with other areas through the infrared pre-acquisition image;

[0065] Taking the center point of the temperature anomaly area as the center seed point, a surface image segment of a preset size is divided;

[0066] The following steps are repeated until the surface image of the weld is completely divided into multiple segments:

[0067] A central seed point is randomly set in the undivided remaining area of ​​the surface image, and the division is continued towards the undivided remaining area until the shape size defined by each segment is reached.

[0068] Subsequently, the infrared pre-acquisition image is divided according to the division points in the surface image, so as to divide the infrared pre-acquisition image into segments corresponding to the surface image, thereby obtaining the surface image and the infrared pre-acquisition image of each segment.

[0069] S102: Generate a thermal excitation parameter time series variation sequence for each segment using a preset heating parameter time series generation model according to the surface image of each segment and the infrared pre-acquisition image.

[0070] Specifically, the heating parameter time series generation model is trained by the following steps:

[0071] Reference Figure 1 , constructing the heating parameter timing generation model, the heating parameter timing generation model includes a timing generation model input layer, a feature extraction layer, a feature decoupler layer, a timing encoder layer, a timing attention mechanism fusion layer, a heating parameter sequence generation module layer and a timing generation model output layer;

[0072] The input layer of the temporal generation model is used to receive the infrared pre-acquisition image and the surface image of each segment;

[0073] The feature extraction layer is connected to the temporal generation model input layer to obtain the infrared pre-acquisition image and the surface image of each segment. The feature extraction layer includes an infrared pre-acquisition image feature extractor and a surface defect feature extractor. The infrared pre-acquisition image feature extractor is a first deep convolution module. The first deep convolution module is composed of multiple convolution modules and pooling modules to obtain the feature representation F of the infrared pre-acquisition image through multiple layers of convolution. heat The surface defect feature extractor is a second deep convolution module, which is composed of multiple convolution modules and pooling modules to obtain the surface feature representation F of the surface image through multi-layer convolution. surface ;

[0074] The feature decoupler layer is connected to the feature extraction layer to obtain the feature representation F of the infrared pre-acquisition image. heat and surface characteristics F surface The feature decoupler layer includes a feature decoupler input layer, a decoupled convolutional neural network, and a feature decoupler output layer. The feature decoupler input layer obtains the surface feature representation F surface And input it into the decoupled convolutional neural network to obtain the feature representation F of the infrared pre-acquired image heat Dimensionally consistent surface defects affect the representation of features F surface-heat , the feature decoupler output layer obtains the feature representation F of the infrared pre-acquisition image heat And the feature is represented by F heat Corresponding to the subtraction of surface defect influence feature F surface-heat In this way, we get the internal feature representation F internal.

[0075] Specifically, the decoupled convolutional neural network is obtained by the following steps:

[0076] Collecting a plurality of surface images with only surface defects and corresponding infrared pre-acquisition images;

[0077] The surface image is processed into the feature representation F of the infrared pre-acquisition image through the same model structure as the second deep convolution module heat ;

[0078] The infrared pre-acquired image is processed into a surface feature representation F through the same model structure as the first deep convolution module. surface ;

[0079] The convolutional neural network is used as the network framework of the decoupled convolutional neural network, and the surface features are used to represent F surface As the input of the convolutional neural network, the feature representation F of the infrared pre-acquired image heat As the label, the first loss function is used as the loss function, and the decoupled convolutional neural network is trained in this way;

[0080] The first loss function is:

[0081]

[0082] Among them, surface defects affect the characteristic F surface-heat is the prediction output of the decoupled convolutional neural network.

[0083] The temporal encoder layer encodes each time step according to the preset total heating time T and the preset time step t to obtain the time encoding vector E t ;

[0084] Among them, the total heating time T and the time step t are artificially set values, and the time encoding vector E t It actually represents a period of time. If the time step is 1, E1 represents the period of 0-1 seconds.

[0085] The temporal attention mechanism fusion layer includes a weight calculation module and a feature fusion module. The weight calculation module is a first multi-layer perceptron. The first multi-layer perceptron is used to calculate the time encoding vector E according to the time encoding vector E. t Generate the surface feature weight α corresponding to each time step t and the internal feature weight β t ;

[0086] The feature fusion module is used to calculate the surface feature weight α t and the internal feature weight β t The surface feature Fsurface and the internal feature representation F internal Fusion is performed to obtain the fusion feature F of time step t fused，t ;

[0087] A heating parameter sequence generation module layer, wherein the heating parameter sequence generation module layer comprises a sequence input layer and a first Transformer decoder, wherein the sequence input layer is used to receive a fusion feature sequence (F fused，1 , F fused，2 , F fused，3 , ..., F fused，T ) and the sequence of time-encoded vectors (E1, E2, E3, …, E T ), the first Transformer decoder receives the fused feature sequence and the time encoding vector sequence through the sequence input layer to generate a thermal excitation parameter temporal change sequence;

[0088] The time series generation model output layer receives the output of the heating parameter sequence generation module layer to output the thermal excitation parameter time series change sequence to the outside.

[0089] A plurality of thermal excitation parameter time series change sequences and corresponding surface images and infrared pre-acquisition images are collected as training sets, and the surface images and infrared pre-acquisition images are used as inputs of the heating parameter time series generation model, and the corresponding thermal excitation parameter time series change sequences are used as training labels to train and obtain the heating parameter time series generation model.

[0090] Specifically, the training of the heating parameter time series generation model is divided into three steps:

[0091] In the first step, the feature decoupling module is first trained. The training method is described in the above embodiment to obtain the model weights in the feature decoupler.

[0092] In the second step, the heating parameter generation module and the temporal attention mechanism fusion layer are trained, the model weights of the feature decoupler are frozen, and the real preferred thermal excitation parameter temporal change sequence is used as the label, the surface image and the infrared pre-acquisition image are used as the input, and the second loss function is used as the loss function to train the heating parameter generation module and the temporal attention mechanism fusion layer. The second loss function is:

[0093]

[0094] in, is the parameter corresponding to the tth time step in the real time series of thermal excitation parameters, It is the parameter corresponding to the tth time step in the predicted thermal excitation parameter time series change sequence.

[0095] The third step is joint training. All model weights are unfrozen and the joint loss is used as the loss function. The joint loss function is the sum of the first loss function and the second loss function. The real preferred thermal excitation parameter time series change sequence is still used as the label, and the surface image and infrared pre-acquisition image are used as input to train all model weights.

[0096] In another embodiment, it is also possible to select conventional artificial intelligence models such as convolutional neural networks and other models based only on the data set of surface image-thermal excitation parameter time series change sequence, and train them to generate thermal excitation parameter time series change sequence directly based on the surface image, but the effect of this approach is inferior to the aforementioned method in the embodiment of the present application.

[0097] S103: Thermally excite and heat the weld area of ​​each segment according to the time series change sequence of the thermal excitation parameters of each segment.

[0098] Specifically, according to the time series change sequence of the thermal excitation parameters of each segment obtained in S102, the thermal excitation device is controlled to perform thermal excitation heating on the bad weld area of ​​each segment.

[0099] S104: During the thermal excitation heating process, a plurality of infrared images of each segment along the time sequence are acquired by an infrared camera to obtain a time sequence change sequence of infrared images of each segment.

[0100] S105: Inspect the weld quality of each segment using a preset weld quality inspection model according to the time series variation sequence of the infrared images of each segment.

[0101] Specifically, the weld quality detection model is trained by the following steps:

[0102] Long short-term memory network is used as the network framework of weld quality detection model;

[0103] Multiple infrared image time series change sequences and the corresponding quality scores are collected as the training set of the weld quality detection model. The infrared image time series change sequence is used as the input of the long short-term memory network, and the corresponding quality score is used as the label of the long short-term memory network to train the weld quality detection model.

[0104] The beneficial effects and working principles of the weld quality detection method based on machine vision provided in the embodiment of the present application are discussed as follows:

[0105] One of its beneficial effects and the discussion of its working principle is that the present application can use a visible light camera to obtain the surface image of the weld, and for each segment of the surface image, a preset heating parameter timing generation model is used to generate a corresponding thermal excitation parameter timing change sequence. Specifically, this is because different surface defects will affect the heat conduction path inside the weld. By machine vision to identify the location, shape and type of surface defects, the model can adjust the heating parameters, such as heating power, frequency and duration, according to these characteristics, so that heat is more effectively transferred to specific defective areas. This targeted heating can produce a more significant temperature difference between the defective area and the surrounding normal area, enhancing the thermal contrast. Moreover, when surface defects and internal defects exist at the same time, the surface defects are often associated with the internal defects. Through this correlation feature between surface defects and internal defects, the model can also strengthen the thermal response of internal defects.

[0106] Subsequently, the present application can perform thermal excitation heating on the weld area of ​​each segment according to the generated thermal excitation parameter time series change sequence. In this process, multiple infrared images along the time series of each segment are acquired by an infrared camera to form a time series change sequence of infrared images. The infrared images contained in the sequence not only enhance the thermal response of known defects, but also capture the thermal characteristics of other possible internal defects due to the improved overall thermal contrast caused by the optimized heating process.

[0107] Finally, based on the temporal change sequence of infrared images in each segment, the preset weld quality inspection model is used to conduct a comprehensive inspection of the weld quality. This model can accurately identify quality problems in welds by analyzing the temporal and spatial characteristics of infrared images.

[0108] Therefore, the weld quality inspection method based on machine vision provided in this application can improve the accuracy of weld quality inspection.

[0109] The second beneficial effect and its working principle are discussed as follows: stress elimination by heat treatment of welds is a common processing step. The present application achieves two effects at the same time by using an infrared camera to obtain an infrared pre-acquisition image of each segment after heat treatment and before obtaining a surface image. The first effect is that the temperature field characteristics of the weld after heat treatment can be used to quickly identify abnormal temperature areas on the surface and inside of the weld. Through these infrared pre-acquisition images, the complete position of surface defects or internal defects can be quickly located, avoiding the occurrence of segment division errors in the subsequent image segmentation process. This pre-scanning method effectively reduces the detection error caused by division errors and improves the accuracy of detection.

[0110] The second effect is that since both the surface defects and internal defects of the weld will have different effects on the thermal conduction and thermal radiation characteristics, when the surface defects and internal defects are superimposed, the heating curve generated directly based on the surface image does not take into account the thermal response of the internal defects, which affects the detection results. In addition, excessive reliance on surface features to generate heating curves will lead to overfitting of the model, that is, the generated thermal excitation parameter time series will be more prominent in expressing the thermal response caused by surface defects, resulting in internal defects being concealed to a certain extent.

[0111] By acquiring an infrared pre-acquisition image before the complete weld inspection, the present application can obtain the temperature distribution information of the weld before the formal inspection, thereby introducing part of the internal defect information and assisting in determining the best heating parameter combination. Therefore, the weld quality inspection method based on machine vision provided by the present application can improve the accuracy of weld quality inspection.

[0112] The third beneficial effect and its working principle are as follows: the weld quality detection method based on machine vision provided by the present application introduces a feature decoupling mechanism when constructing a heating parameter timing generation model, and separates the superimposed thermal features into independent surface features and internal features. Through the feature decoupler layer, a decoupled convolutional neural network is used to remove the influence of surface defects from the feature representation of the infrared pre-acquired image to obtain a pure internal feature representation. This process ensures that the features of surface defects and internal defects can be identified and analyzed separately, avoiding interference between each other and improving the accuracy of defect detection.

[0113] On this basis, the present application further combines a temporal attention mechanism module to dynamically adjust the attention to surface features and internal features according to different time series during the heating process. In the early stage of heating, the heat is mainly concentrated in the surface area of ​​the material, and the thermal response of surface defects is more obvious. Therefore, the temporal attention mechanism gives higher weights to surface features at this stage. As the heating progresses, the heat gradually conducts into the interior of the material, the thermal response of internal defects begins to increase, and the temporal attention mechanism gradually increases the weight of internal features.

[0114] This dynamic adjustment strategy enables the system to generate more appropriate heating parameters at different heating stages and specifically enhance the thermal contrast of different defects. Through the heating parameter sequence generation module layer, according to the fusion features along the time series output by the time series attention mechanism fusion layer, the model can generate a time series change sequence of thermal excitation parameters that meets actual needs. This sequence is optimized at each time step to ensure effective detection of surface and internal defects.

[0115] Therefore, the weld quality inspection method based on machine vision provided in this application can improve the accuracy of weld quality inspection.

[0116] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In addition, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. Moreover, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can represent A or B; "and / or" in this article is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. Furthermore, in the description of the embodiments of the present application, “plurality” refers to two or more than two.

[0117] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A weld quality detection method based on machine vision, characterized in that: The following steps are involved: Acquire a surface image of the weld by using a visible light camera, and divide the surface image of the weld into a plurality of segments; According to the surface image of each segment, a time series variation sequence of thermal excitation parameters of each segment is generated by a preset heating parameter time series generation model; Thermally excitation heating is performed on the weld area of ​​each segment according to the time series variation sequence of thermal excitation parameters of each segment; During the thermal excitation heating process, a plurality of infrared images along the time sequence of each segment are acquired by an infrared camera to obtain a time sequence change sequence of infrared images of each segment; According to the time series change sequence of the infrared images of each segment, the weld quality of each segment is detected by a preset weld quality detection model; The weld quality detection method based on machine vision also includes the following steps: A weld heat treatment step is provided before the surface image of the weld is acquired by the camera; After the weld is heat treated and before the surface image of the weld is acquired by the visible light camera, an infrared pre-acquisition image of each segment is acquired by the infrared camera; Furthermore, according to the infrared pre-acquired image of each segment, a time series variation sequence of thermal excitation parameters of each segment is generated through a preset heating parameter time series generation model; The heating parameter time series generation model is trained by the following steps: Constructing the heating parameter timing generation model, the heating parameter timing generation model includes a timing generation model input layer, a feature extraction layer, a feature decoupler layer, a timing encoder layer, a timing attention mechanism fusion layer, a heating parameter sequence generation module layer and a timing generation model output layer; The input layer of the temporal generation model is used to receive the infrared pre-acquisition image and the surface image of each segment; The feature extraction layer is connected to the temporal generation model input layer to obtain the infrared pre-acquisition image and the surface image of each segment. The feature extraction layer includes an infrared pre-acquisition image feature extractor and a surface defect feature extractor. The infrared pre-acquisition image feature extractor is a first deep convolution module. The first deep convolution module is composed of multiple convolution modules and pooling modules to obtain the feature representation F of the infrared pre-acquisition image through multiple layers of convolution. heat The surface defect feature extractor is a second deep convolution module, which is composed of multiple convolution modules and pooling modules to obtain the surface feature representation F of the surface image through multi-layer convolution. surface ; The feature decoupler layer is connected to the feature extraction layer to obtain the feature representation F of the infrared pre-acquisition image. heat and surface characteristics F surface The feature decoupler layer includes a feature decoupler input layer, a decoupled convolutional neural network, and a feature decoupler output layer. The feature decoupler input layer obtains the surface feature representation F surface And input it into the decoupled convolutional neural network to obtain the feature representation F of the infrared pre-acquired image heat Dimensionally consistent surface defects affect the representation of features F surface-heat , the feature decoupler output layer obtains the feature representation F of the infrared pre-acquisition image heat And the feature is represented by F heat Subtract surface defects affecting feature F surface-heat In this way, we get the internal feature representation F internal ; The temporal encoder layer encodes each time step according to the preset total heating time T and the preset time step t to obtain the time encoding vector E t ; The temporal attention mechanism fusion layer includes a weight calculation module and a feature fusion module. The weight calculation module is a first multi-layer perceptron. The first multi-layer perceptron is used to calculate the time encoding vector E according to the time encoding vector E. t Generate the surface feature weight α corresponding to each time step t and the internal feature weight β t ; The feature fusion module is used to calculate the surface feature weight α t and the internal feature weight β t The surface feature F surface and the internal feature representation F internal Fusion is performed to obtain the fusion feature F of time step t fused，t ; A heating parameter sequence generation module layer, wherein the heating parameter sequence generation module layer comprises a sequence input layer and a first Transformer decoder, wherein the sequence input layer is used to receive a fusion feature sequence (F fused，1 , F fused，2 , F fused，3 , ..., F fused，T ) and the sequence of time-encoded vectors (E1, E2, E3, …, E T ), the first Transformer decoder receives the fused feature sequence and the time encoding vector sequence through the sequence input layer to generate a thermal excitation parameter temporal change sequence; A time series generation model output layer, wherein the time series generation model output layer receives the output of the heating parameter sequence generation module layer to output a thermal excitation parameter time series change sequence to the outside; A plurality of thermal excitation parameter time series change sequences and corresponding surface images and infrared pre-acquisition images are collected as training sets, and the surface images and infrared pre-acquisition images are used as inputs of the heating parameter time series generation model, and the corresponding thermal excitation parameter time series change sequences are used as training labels to train and obtain the heating parameter time series generation model.

2. The weld quality detection method based on machine vision according to claim 1 is characterized in that: Dividing the surface image of the weld into a plurality of segments comprises the following steps: Acquire a temperature abnormality area in the infrared pre-acquisition image whose temperature is abnormal compared with other areas through the infrared pre-acquisition image; Taking the center point of the temperature anomaly area as the center seed point, a surface image segment of a preset size is divided; The following steps are repeated until the surface image of the weld is completely divided into multiple segments: A central seed point is randomly set in the undivided remaining area of ​​the surface image, and the division is continued towards the undivided remaining area until the shape size defined by each segment is reached.

3. The weld quality detection method based on machine vision according to claim 1 is characterized in that: The decoupled convolutional neural network is obtained by the following steps: Collecting a plurality of surface images with only surface defects and corresponding infrared pre-acquisition images; The surface image is processed into the feature representation F of the infrared pre-acquisition image through the same model structure as the second deep convolution module heat ; The infrared pre-acquired image is processed into a surface feature representation F through the same model structure as the first deep convolution module. surface ; The convolutional neural network is used as the network framework of the decoupled convolutional neural network, and the surface features are used to represent F surface As the input of the convolutional neural network, the feature representation F of the infrared pre-acquired image heat As the label, the first loss function is used as the loss function, and the decoupled convolutional neural network is trained in this way; The first loss function is: Among them, surface defects affect the characteristic F surface-heat is the prediction output of the decoupled convolutional neural network.

4. The weld quality detection method based on machine vision according to claim 1 is characterized in that: The weld quality detection model is trained by the following steps: Long short-term memory network is used as the network framework of weld quality detection model; Multiple infrared image time series change sequences and the corresponding quality scores are collected as the training set of the weld quality detection model. The infrared image time series change sequence is used as the input of the long short-term memory network, and the corresponding quality score is used as the label of the long short-term memory network to train the weld quality detection model.

Images