Welding quality detection method and device, electronic equipment and storage medium

Abstract

The invention provides a welding quality detection method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps of obtaining multi-source data collected for a welding area in the welding process, wherein the multi-source data comprises data collected by different types of sensors; inputting the multi-source data into a pre-trained welding quality detection model to obtain probabilities of a plurality of welding defect categories; under the condition that the maximum probability exceeds a probability threshold value, it is determined that the welding quality detection result of the welding area is abnormal welding quality, and the welding defect type corresponding to the maximum probability serves as the defect type of the welding area; and determining a defect position corresponding to the defect type through bounding box regression or feature map coordinate mapping. Multi-source data are analyzed through artificial intelligence, the welding quality can be rapidly and accurately recognized in real time, the welding quality can be improved, and then the production cost and the product reject ratio are reduced to meet the large-scale and high-precision production requirements.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a welding quality detection method, device, electronic device and storage medium. Background Art

[0002] Currently, for the welding quality detection in welding operations, it mainly relies on manual sampling inspection or off-line inspection. Among them, manual sampling inspection depends on the experience of workers, has strong subjectivity and low efficiency, and it is difficult to detect internal defects. Off-line inspection (such as destructive inspection, ultrasonic inspection, etc.) will interrupt the welding process, cannot detect in real time and may damage the product. These methods will all lead to problems such as low welding detection efficiency, inability to correct errors in time, and increased production costs.

[0003] In response to the above problems, online welding quality detection methods have been proposed currently. However, the current online welding quality detection methods all detect through a single sensor, cannot comprehensively reflect the welding quality, and are prone to missed detection. These shortcomings will lead to unstable welding quality, increased production costs and product unqualified rate, and it is difficult to meet the production requirements of large scale and high precision. Summary of the Invention

[0004] Based on the above technical problems, the present invention provides a welding quality detection method, device, electronic device and storage medium, aiming to overcome or at least partially solve the above problems.

[0005] The present invention provides a welding quality detection method, and the method includes: Obtain multi-source data collected for a welding area during the welding process, where the multi-source data includes data collected by different types of sensors; the multi-source data includes: visual images and temperature distribution data; Input the multi-source data into a pre-trained welding quality detection model to obtain probabilities of multiple welding defect categories; In the case where the maximum probability exceeds a probability threshold, determine that the welding quality detection result of the welding area is abnormal welding quality, and use the welding defect category corresponding to the maximum probability as the defect type of the welding area; Determine the defect position corresponding to the defect type through bounding box regression or feature map coordinate mapping; Wherein, the pre-trained welding quality detection model is trained based on sample multi-source data collected for a sample welding area during a historical welding process and the true defect information of the sample welding area, and the true defect information is the defect type and defect position manually marked for the sample welding area.

[0006] The second aspect of the present invention provides a welding quality detection system, and the system at least includes: different types of sensors and an embedded control system; The different types of sensors are used to collect multi-source data for the welding area during the welding process and send the multi-source data to the embedded control system; the multi-source data includes: visual images and temperature distribution data; The embedded control system is used to obtain the multi-source data, input the multi-source data into a pre-trained welding quality detection model to obtain the probabilities of multiple welding defect categories; in the case that the maximum probability exceeds the probability threshold, determine that the welding quality detection result of the welding area is abnormal welding quality, and use the welding defect category corresponding to the maximum probability as the defect type of the welding area; determine the defect position corresponding to the defect type through bounding box regression or feature map coordinate mapping; Wherein, the pre-trained welding quality detection model is trained based on the sample multi-source data collected for the sample welding area during the historical welding process and the true defect information of the sample welding area, and the true defect information is the defect type and defect position manually marked for the sample welding area.

[0007] A third aspect of the present invention provides an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, it implements the welding quality detection method according to the first aspect of the present invention.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the welding quality detection method according to the first aspect of the present invention.

[0009] In the welding quality detection method provided by the present invention, multi-source data is used to complement each other in multiple dimensions, which can reflect the welding quality in multiple aspects, avoid the limitations and missed detections of single-sensor detection, improve the accuracy of welding quality detection, and the pre-trained welding quality detection model in the present invention learns a large amount of sample multi-source data and corresponding true defect information, and can quickly and accurately identify the defect type and defect position of the welding area, and then determine the welding detection quality result of the welding area, thereby improving the welding quality, reducing the production cost and the product unqualified rate to meet the large-scale and high-precision production requirements. In addition, the present invention can detect the welding quality online during the welding process, realizes automatic and intelligent detection, reduces the workload of manual detection, reduces the labor cost, reduces the influence of human factors, and improves the reliability and consistency of the welding quality detection results. Description of the Drawings

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for the description of the embodiments of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0011] Figure 1 is a flowchart of a welding quality detection method shown in an embodiment of the present invention; Figure 2 is a schematic diagram of visual sensor signal processing and transmission shown in an embodiment of the present invention; Figure 3 is a schematic diagram of infrared thermal imager signal processing and transmission shown in an embodiment of the present invention; Figure 4 is a schematic diagram of an infrared thermal imager component and a visual sensor component shown in an embodiment of the present invention; Figure 5 is a schematic diagram of an embedded control system shown in an embodiment of the present invention; Figure 6 is a schematic diagram of a laser welding robot adaptation module shown in an embodiment of the present invention; Figure 7 is a schematic diagram of a parameter adjustment process shown in an embodiment of the present invention; Figure 8 is a schematic diagram of signal output between a host computer and an embedded control system shown in an embodiment of the present invention; Figure 9 is a schematic diagram of an on-line welding quality detection method based on laser welding shown in an embodiment of the present invention; Figure 10 is a schematic diagram of an on-line welding quality detection method based on laser welding shown in an embodiment of the present invention; Figure 11 is a structural block diagram of a welding quality detection system provided by an embodiment of the present invention; Figure 12 is a schematic diagram of an electronic device shown in an embodiment of the present invention. Detailed implementation manners

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0013] Refer to Figure 1 ,Figure 1 is a flowchart of a welding quality detection method shown in an embodiment of the present invention. As Figure 1 shown, the welding quality detection method may include the following steps: Step S11: Obtain multi-source data collected for the welding area during the welding process, where the multi-source data includes data collected by different types of sensors; the multi-source data includes: visual images and temperature distribution data.

[0014] In this embodiment, during the welding process of welding operations, multi-source data collected for the welding area can be obtained, where the multi-source data includes data collected by different types of sensors. In this embodiment, multi-source data of the welding area during the welding process can be collected through multiple sensors of different types to monitor the dimensional data of different aspects of the welding area in real time. Among them, the multi-source data of this embodiment at least includes: visual images and temperature distribution data.

[0015] Step S12: Input the multi-source data into a pre-trained welding quality detection model to obtain probabilities of multiple welding defect categories.

[0016] In this embodiment, the obtained multi-source data can be input into a pre-trained welding quality detection model to obtain probabilities of multiple welding defect categories corresponding to the welding area output by the pre-trained welding quality detection model. The pre-trained welding quality detection model of this embodiment is trained based on the sample multi-source data collected for the sample welding area during the historical welding process and the true defect information of the sample welding area. The pre-trained welding quality detection model can judge the welding defects in the welding area and output probabilities of multiple welding defect categories, that is, obtain the probability value corresponding to each welding defect category in the multiple welding defect categories. For example, the defect categories include but are not limited to: pores, cracks, lack of fusion, slag inclusions, undercut, etc., and there is no limitation on this. And, for example, the probabilities of the multiple welding defect categories output include: the probability of pores is 90%, the probability of cracks is 5%, etc.

[0017] Among them, the sample welding area is the welding area targeted during the training process of the welding quality detection model, the sample multi-source data is the multi-source data corresponding to the sample welding area, such as the sample visual image and sample temperature distribution data corresponding to the sample welding area, and the true defect information is the defect type and defect location manually marked for the sample welding area.

[0018] Step S13: When the maximum probability exceeds the probability threshold, determine that the welding quality detection result of the welding area is abnormal welding quality, and use the welding defect category corresponding to the maximum probability as the defect type of the welding area.

[0019] In this embodiment, after the pre-trained welding quality detection model obtains the probabilities of multiple welding defect categories, it can determine the maximum probability therefrom and determine whether the maximum probability exceeds a probability threshold (which can be freely set in advance). When it is determined that the maximum probability exceeds the probability threshold, it is determined that the welding quality detection result of the welding area is abnormal welding quality, and the welding defect category corresponding to the maximum probability is used as the defect type of the welding area. For example, if it is determined that the probability of porosity defect (such as 90%) is the largest and exceeds the probability threshold (such as 70%), it is determined that there is a porosity defect in the welding area, the defect type of the welding area is determined to be porosity defect, and the corresponding welding quality detection result is output.

[0020] Step S14: Determine the defect position corresponding to the defect type through bounding box regression or feature map coordinate mapping.

[0021] In this embodiment, after the pre-trained welding quality detection model obtains the defect type of the welding area, it can determine the defect position (such as the coordinates of a certain area of the weld) corresponding to the defect type of the welding area through bounding box regression (such as YOLOv5) or feature map coordinate mapping. That is to say, when the pre-trained welding quality detection model in this embodiment determines that the welding quality detection result is abnormal welding quality, the welding quality detection result output by the pre-trained welding quality detection model at least includes: defect type and defect position.

[0022] In this embodiment, an artificial intelligence algorithm is introduced to analyze the multi-sensor data fusion (visual image and temperature distribution data), establish an association model (i.e., the pre-trained welding quality detection model) between the welding quality and the multi-source data (visual image and temperature distribution data), and realize the real-time diagnosis of the defect type and defect position of the welding area. Among them, the use of multi-source data to complement each other in multiple dimensions can reflect the welding quality in multiple aspects, avoid the limitations and missed detections of single-sensor detection, improve the accuracy of welding quality detection, and the pre-trained welding quality detection model in the present invention has learned a large number of sample multi-source data and corresponding real defect information, and can quickly and accurately identify the defect type and defect position of the welding area, and then determine the welding detection quality result of the welding area, so as to improve the welding quality, and further reduce the production cost and product unqualified rate to meet the large-scale and high-precision production requirements. In addition, the present invention can detect the welding quality online during the welding process, realizes automatic and intelligent detection, reduces the workload of manual detection, reduces the labor cost, reduces the influence of human factors, and improves the reliability and consistency of the welding quality detection results. Moreover, the multi-sensor fusion and artificial intelligence analysis adopted in this embodiment have strong compatibility and adaptability, and can be applied to different types of welding and different welding materials and processes, so as to realize the application in a variety of welding scenarios.

[0023] Combined with the above embodiments, in one implementation, the present invention further provides a welding quality detection method. In this method, in addition to the above steps, it further includes step S21: Step S21: In the case where the welding quality detection result is abnormal welding quality, adjust the welding parameters, and weld the welding area according to the adjusted welding parameters.

[0024] In this embodiment, the welding quality detection result can be divided into abnormal welding quality or normal welding quality. In the case where the welding quality detection result output by the pre-trained welding quality detection model is abnormal welding quality, the current welding parameters during the welding process can be adjusted to obtain the adjusted welding parameters, and then the welding area can be welded according to the adjusted welding parameters. In an optional implementation, the welding parameters at least include any one or more of the following: laser power, welding speed, welding path.

[0025] In this embodiment, it is possible to monitor the welding quality situation during the welding process in real time. Once abnormal welding quality is detected, the welding parameters will be automatically adjusted in real time to form a closed-loop control for welding, further ensuring the welding quality. Moreover, the real-time monitoring and closed-loop control mechanism in this embodiment can make the welding process more stable and efficient, reducing production pauses and adjustment times caused by welding quality problems, thereby improving production efficiency.

[0026] Combined with the above embodiments, in one implementation, in the case where the welding quality detection result output by the pre-trained welding quality detection model is normal welding quality, the welding area can continue to be welded according to the current welding parameters (i.e., the original welding parameters).

[0027] In an optional implementation, in the case where the maximum probability among the probabilities of multiple welding defect categories determined and output by the pre-trained welding quality detection model does not exceed the probability threshold, it can be directly determined that the welding quality detection result of this welding area is normal welding quality, or, in the case where it is determined that the maximum probability does not exceed the probability threshold, it is further analyzed to determine whether the welding quality detection result is normal welding quality.

[0028] Combined with the above embodiments, in one implementation, the present invention further provides a fatigue analysis method. In this method, the above step S12 can specifically include steps S31 to S34: Step S31: Analyze the visual image to obtain the molten pool morphology information, weld width information, and weld depth information.

[0029] In this embodiment, the visual image can be data collected for the welding area by a visual sensor (such as a high-precision visual sensor, such as a CCD camera), and the temperature distribution data can be data collected for the welding area by an infrared thermal imager. The welding area in this embodiment at least includes: a molten pool; this embodiment can analyze the visually acquired images in real time to obtain molten pool morphology information, weld width information, and weld depth information.

[0030] Step S32: Convert the temperature distribution data to obtain a thermal map.

[0031] In this embodiment, the temperature distribution data can be converted into an image form to obtain a thermal map.

[0032] Step S33: Stitch the visual image carrying the molten pool morphology information, the weld width information, and the weld depth information with the thermal map representing the temperature distribution data to obtain a stitched image.

[0033] In this embodiment, the visual image carrying the molten pool morphology information, the weld width information, and the weld depth information can be stitched with the thermal map representing the temperature distribution data to obtain a stitched image.

[0034] Step S34: Input the stitched image into the pre-trained welding quality detection model to obtain the probabilities of the multiple welding defect categories.

[0035] In this embodiment, after obtaining the stitched image, the stitched image can be input into the pre-trained welding quality detection model to obtain the probabilities of the multiple welding defect categories corresponding to the welding area output by the pre-trained welding quality detection model.

[0036] In this embodiment, a multi-sensor fusion method using a visual sensor and an infrared thermal imager is adopted. The visual sensor can accurately capture the geometric features of the weld, while the infrared thermal imager can accurately monitor the temperature change in the welding area. The two types of data complement each other, avoiding the limitations of single-sensor detection and greatly improving the accuracy of welding quality detection.

[0037] Combining the above embodiments, the present invention also provides a welding quality detection method. In this method, "acquiring multi-source data collected for the welding area during the welding process" in step S11 above can specifically include step S41, and "analyzing the visual image to obtain molten pool morphology information and weld depth information" in step S31 above can specifically include steps S42 to S45: Step S41: Project a light source onto the welding area to obtain a visual image collected for the welding area.

[0038] In this embodiment, during the welding process, a vision sensor combined with a light source can be used. By utilizing the principle of structured light, the light source is projected onto the welding area, and the vision sensor collects the visual image of the welding area and obtains the visual image collected for the welding area.

[0039] In an optional example, a high-speed camera combined with a light source of a specific wavelength can be used. The light source of the specific wavelength is projected onto the surface of the welding area, and relevant information of the welding area is obtained by analyzing the distortion of the reflected light. For example, the light source of the specific wavelength can be light of a known structure, such as using a laser as the projected light. Among them, a commonly used one is a semiconductor laser, and its output wavelength is generally in the near-infrared band, such as 808nm, 980nm, etc. The laser in this band can not only meet the requirements of structured light projection but also interact well with the surface of the welding area, and has a high matching degree with the photosensitive characteristics of the high-speed camera, which is convenient for obtaining a clear visual image (i.e., the reflected light image).

[0040] Step S42: Analyze the visual image to determine the distortion information of the reflected light generated by the projection of the light source onto the welding area. The distortion information at least includes: geometric distortion information and gray value distortion information.

[0041] The molten pool morphology information in this embodiment at least includes: the concave-convex information, depth information, width information, and surface physical information of the molten pool. In this embodiment, the obtained visual image can be analyzed to determine the distortion information of the reflected light generated by the projection of the light source onto the welding area. The distortion information of the reflected light at least includes: geometric distortion information and gray value distortion information.

[0042] Among them, the geometric distortion information is the information of geometric shape distortion. The geometric shape distortion is: when the structured light is projected onto the surface of the molten pool, due to the three-dimensional undulation of the molten pool, the propagation path of the reflected light in space changes, resulting in the bending and distortion of the originally regular structured light pattern (such as stripes, grids, etc.) on the imaging plane. For example, for the originally parallel stripes, at the convex part of the molten pool, the stripe spacing of the reflected light imaging will become smaller, showing a shape of bending inward; while at the concave part of the molten pool, the stripe spacing will become larger and bend outward.

[0043] The gray value distortion information is the information of gray value distortion. The gray value distortion is: the physical characteristics of the molten pool surface (such as uneven temperature distribution, metal vapor, etc.) will affect the intensity of the reflected light, thereby causing the distortion of the gray value of the reflected light.

[0044] Step S43: Based on the geometric distortion information, determine the three-dimensional morphology information of the molten pool.

[0045] In this embodiment, the three-dimensional morphology information of the molten pool can be determined based on the geometric distortion information.

[0046] Step S44: Based on the three-dimensional topography information, determine the weld depth information, the concavity and convexity information, and the depth information of the molten pool.

[0047] In this embodiment, after determining the three-dimensional topography information, the concavity and convexity condition and the depth information of the molten pool can be determined based on the three-dimensional topography information. Moreover, based on the three-dimensional topography information and in combination with the triangulation principle, the weld depth information can be calculated.

[0048] Step S45: Based on the grayscale value distortion information, determine the surface physical information and the width information of the molten pool.

[0049] In this embodiment, the surface physical information and the width information of the molten pool can be determined based on the obtained grayscale value distortion information. In the region with a relatively high temperature of the molten pool, the reflectivity of the metal may change, the intensity of the reflected light increases, and the corresponding grayscale value in the image increases; while in the region blocked by metal vapor, the intensity of the reflected light weakens, the grayscale value decreases, and dark spots or shadow regions appear. Thus, the surface physical information of the molten pool can be determined based on the grayscale value distortion information. The surface physical information of the molten pool in this embodiment is the information of the physical characteristics of the molten pool surface, and the physical characteristics of the molten pool surface include but are not limited to uneven temperature distribution and metal vapor.

[0050] For the width information of the molten pool, in this embodiment, based on the edge detection algorithm, the edge can be determined by using the grayscale difference between the molten pool and the surrounding area in the visual image, and then the width information of the molten pool can be calculated.

[0051] In one embodiment, the photoelectric conversion device inside the vision sensor converts the optical signal collected in real time into an electrical signal (analog signal). After preliminary processing such as amplification and filtering, it is converted into a digital signal through an analog-to-digital converter (ADC) for transmission. For the problem of signal failure, in one embodiment, a redundant backup vision sensor can be adopted. When the main vision sensor fails, it automatically switches to the backup vision sensor. For the error problem, in one embodiment, the vision sensor can be calibrated regularly to calibrate the optical system parameters, and a filtering algorithm can be adopted in the data processing stage to remove noise and outliers.

[0052] For example, optoelectronic conversion devices such as Complementary Metal-Oxide-Semiconductor (CMOS) or charge-coupled device (CCD) can be used. Taking the CMOS image sensor as an example, it contains a large number of pixel units inside, and each pixel unit has a photodiode. When the reflected light irradiates the photodiode, photons interact with the semiconductor material to generate electron-hole pairs. According to the photovoltaic effect, a potential difference is formed on both sides of the PN junction of the photodiode, thereby converting the optical signal into a corresponding charge signal, and the intensity of the light determines the amount of charge generated.

[0053] As Figure 2 shown, Figure 2 FIG. is a schematic diagram of visual sensor signal processing and transmission shown in an embodiment of the present invention. In this embodiment, the visual sensor is a CCD camera. The CCD camera collects optical signals in the welding area (such as optical signals of the molten pool morphology, weld width, and depth) during the welding process, and sequentially obtains digital signals through optoelectronic conversion, analog signal processing, and ADC conversion, and then transmits the digital signals to the data processing module of the embedded control system. The embedded control system is used to detect the welding quality.

[0054] In an embodiment, an infrared thermal imager synchronously monitors the infrared radiation signal in the welding area. After the infrared radiation signal is processed, temperature distribution data (i.e., temperature distribution map) of the welding area is formed. The signal processing flow is: infrared radiation signal → analog electrical signal → digital signal → temperature distribution data.

[0055] As Figure 3 shown, Figure 3 FIG. is a schematic diagram of infrared thermal imager signal processing and transmission shown in an embodiment of the present invention. The infrared thermal imager converts the infrared radiation signal in the welding area into an electrical signal (analog signal) through an infrared detector. After signal processing (such as analog signal conditioning), it is converted into a digital signal through ADC and transmitted to the data processing module of the embedded control system. The embedded control system is used to detect the welding quality, and the data processing module analyzes the visual image to obtain molten pool morphology information, weld width information, and weld depth information.

[0056] Combining the above embodiments, the present invention also provides a welding quality detection method. In this method, step S34 above may specifically include steps S51 to S54: Step S51: Extract features of different dimensions from the spliced image through the multi-layer convolutional layer to obtain the first image feature.

[0057] In this embodiment, the pre-trained welding quality detection model is a convolutional neural network, and the pre-trained welding quality detection model at least includes: multiple convolutional layers, pooling layers, and fully connected layers. After inputting the spliced image into the pre-trained welding quality detection model in this embodiment, the pre-trained welding quality detection model first extracts features of different dimensions from the spliced image through multiple convolutional layers to obtain the first image feature. In this process, the molten pool morphology information, weld width information, and weld depth information carried by the visual image can be used as part of the visual features of the visual image, and geometric features are extracted through multiple convolutional layers, so as to finally obtain the first image feature output by the multiple convolutional layers.

[0058] Among them, the convolutional layer extracts image features from the spliced image by sliding and calculating the convolutional kernel to obtain a series of new numerical values, and these numerical values form a new feature map. Different convolutional kernels can extract different features. For example, some convolutional kernels can extract edge features in the image, and some can extract texture features, etc. In this embodiment, through the stacking of multiple convolutional layers, more and more abstract and complex features can be extracted from the original data, so as to obtain the first image feature. For the spliced image (taking a two-dimensional image as an example), assume that the size of the spliced image is H*W (height H and width W), and the size of the convolutional kernel is h*w (height h and width w). During each sliding, the convolutional kernel performs a dot product operation with the pixel values at the corresponding positions on the spliced image, and then adds up all the product results to obtain a new numerical value. The calculation formula for the corresponding area of the convolutional kernel can be: , where s represents the new feature value generated at a specific position after the convolutional operation between the input image (i.e., the spliced image) and the convolutional kernel, which is a refinement and expression of the local features of the data. By calculating a series of s, a new feature map can be formed, and these feature maps retain the features of the input image (molten pool shape, texture, etc.); is the pixel value of the spliced image at position, is the weight value of the convolutional kernel at position.

[0059] In an optional embodiment, the visual image can be preprocessed first to obtain the preprocessed visual image, and then the preprocessed visual image is spliced with the heat map to obtain the spliced image. Among them, the preprocessing in this embodiment includes but is not limited to the following operations: denoising, grayscale transformation, rotation, scaling, noise addition, so as to improve the data quality and enhance the generalization ability of the model.

[0060] Step S52: Fuse the temperature distribution data, the molten pool morphology information, the weld width information, and the weld depth information with the first image feature to obtain the second image feature.

[0061] In this embodiment, after the first image features are extracted, the temperature distribution data, the molten pool morphology information, the weld width information, and the weld depth information can be fused with the first image features to obtain second image features, thereby improving the sensitivity of the welding quality detection model to temperature-related defects, and determining the temperature abnormal area through a threshold to assist in identifying welding defects (such as pores caused by local overheating).

[0062] Step S53: Perform dimensionality reduction processing on the second image features through the pooling layer to obtain welding data features.

[0063] In this embodiment, the second image features are input into the pooling layer, and the pooling layer performs dimensionality reduction processing on the second image features to obtain welding data features. In an optional example, max pooling or average pooling can be used in the pooling layer for pooling processing. For example, average pooling or max pooling: Let the input feature map (i.e., the second image features) be F, with a size of , the pooling kernel size be , and the stride be s; Let the output feature map (i.e., the welding data features) be G, with a size of . For the element at the coordinate in the output feature map G, the calculation formula is: ; where represents the pixel value after offsetting (i, j) within the pooling kernel range starting from the coordinate (m*s, n*s) at an interval of stride s on the input feature map F. By calculating these pixel values, the element value at the corresponding position (m, n) in the output G is obtained. Taking the molten pool image as an example, features such as the gray-scale change and texture details at the molten pool position are crucial for judging welding defects. Through offsetting, the information carried by each pixel within the pooling kernel can be comprehensively considered, such as the gray-scale change feature of the pixels at the molten pool edge, and then a more representative output value can be calculated, enabling the network to better capture the local features during the welding process and improving the accuracy of welding defect recognition.

[0064] Step S54: Process the welding data features through the fully connected layer to obtain the probabilities of the multiple welding defect categories.

[0065] In this embodiment, the feature map obtained after being processed by multiple convolutional layers and pooling layers is unfolded into a one-dimensional vector, and this one-dimensional vector contains the welding data features after being extracted and dimensionally reduced layer by layer. The welding data features are input into the fully connected layer, and the fully connected layer processes the welding data features to obtain the probabilities of the multiple welding defect categories corresponding to this welding area.

[0066] Combined with the above embodiments, the present invention also provides a welding quality detection method. In this method, step S54 above may specifically include step S61: Step S61: Map the welding data features to different welding defect categories through the fully connected layer to obtain the probabilities of the multiple welding defect categories.

[0067] In this embodiment, the fully connected layer can map the extracted features to the welding defect categories. Specifically, each neuron in the fully connected layer of this embodiment is connected to all neurons in the previous layer. These features are linearly transformed through a weight matrix, and then a bias term is added, and then processed through an activation function (such as the Softmax function). For the welding defect judgment task of this embodiment, the Softmax function can map the welding data features to different welding defect categories and output the probabilities of multiple welding defect categories, that is, obtain the probability values corresponding to each welding defect category among the multiple welding defect categories.

[0068] Combined with the above embodiments, in an alternative embodiment, the multi-source data includes visual images and temperature distribution data. When training the welding quality detection model, the training samples are the molten pool images collected in actual welding production (such as including normal welds and defective welds, such as pores, cracks, etc.) and the temperature distribution data of the infrared thermal imager collected synchronously, and the manually marked defect types (such as pores, cracks) and positions (such as weld area coordinates). The temperature abnormal areas can be marked through threshold analysis. The basic network used when training the welding quality detection model can use ResNet-50 or YOLOv5, or other lightweight improved classical convolutional neural networks; after obtaining the results output by the basic model, calculate the loss between the output results and the manually marked defect types and / or defect positions: if the defect type is multi-classification (such as pores, cracks, lack of fusion, etc.), the cross-entropy loss function can be used; if the defect position needs to be output, the mean square error (MSE) loss (regression coordinates) can be combined; finally, the classification and localization losses can be optimized simultaneously, such as using a weighted combined loss function.

[0069] In this way, the input of the trained welding quality detection model is: the molten pool image (including geometric distortion and grayscale distortion information), and the temperature distribution data is converted into an image form (such as a heat map), which is input into the welding quality detection model after being spliced with the molten pool image. The output of the trained welding quality detection model is: quality judgment, defect classification, and defect localization. Among them, defect classification: output the defect type probability through the Softmax function (such as pore probability 90%, crack probability 5%, etc.); defect localization: output the defect position (such as the coordinates of a certain area of the weld) through bounding box regression (such as YOLOv5) or feature map coordinate mapping; quality judgment: judge whether there is a defect through a probability threshold, and if so, output the specific type and position.

[0070] Combined with the above embodiments, in one implementation, an on-line welding quality detection method based on laser welding is proposed. In this method, a vision sensor (such as a high-precision vision sensor) and an infrared thermal imager are used to collect the molten pool morphology, weld width and depth data, and the temperature distribution data of the welding area during the welding process respectively, so as to realize multi-source data fusion monitoring. Artificial intelligence (i.e., a pre-trained welding quality detection model) is introduced to analyze the collected data, establish a correlation model between the welding quality and these data, and realize the real-time diagnosis of welding defects. The embedded control system adjusts the laser power, welding speed or path of the laser welding in real time according to the welding quality analysis result, forms a closed-loop control, and ensures the welding quality.

[0071] In this embodiment, before the on-line welding quality detection, the following preparatory work can be done: The vision sensor and the infrared thermal imager are firmly installed at the end of the laser welding robot through a suitable mounting bracket, ensuring that the measurement field of view completely covers the welding area and adjusting the installation angle and height of the sensor so that it can accurately collect the required data. Use a data transmission line to connect the vision sensor and the infrared thermal imager to the embedded control system to ensure stable data transmission. At the same time, the embedded control system is communicatively connected to the laser welding robot adapter module and the upper computer. And, parameter settings are carried out on the vision sensor and the infrared thermal imager, such as the resolution and frame rate of the vision sensor, the temperature measurement range of the infrared thermal imager, etc. In the embedded control system, relevant parameters of the welding quality detection model and the welding quality judgment threshold are set, and there are corresponding feedback sections for the vision sensor and the infrared thermal imager in this embedded control system.

[0072] As Figure 4 shown, Figure 4 is a schematic diagram of the infrared thermal imager component and the vision sensor component shown in an embodiment of the present invention. In Figure 4 it, the upper part is the infrared thermal imager component, and this infrared thermal imager component includes: an infrared thermal imager, mounting accessories and a calibration position. The lower part is the high-precision vision sensor component, and this high-precision vision sensor component includes: a vision sensor, a mounting bracket and a data transmission line, and this data transmission line is connected to the embedded control system.

[0073] During welding, the vision sensor and the infrared thermal imager can collect multi-source data for the welding area in real time and send the multi-source data to the embedded control system. As Figure 5 shown, Figure 5 is a schematic diagram of the embedded control system shown in an embodiment of the present invention. In Figure 5In this system, the embedded control system includes a data processing module, an AI analysis module, a control instruction generation module, and a communication module. After multi-source data is input into the embedded control system, the data processing module can analyze the visual image to obtain information such as the molten pool morphology, weld width, and depth. Then, the visual image carrying information such as the molten pool morphology, weld width, and depth, and the temperature distribution data are input into the AI analysis module in the embedded control system. The AI analysis module performs welding quality detection through a pre-deployed welding quality detection model to obtain the welding quality detection result. When the AI analysis module determines that the welding quality is abnormal, it sends a signal to the control instruction generation module. The control instruction generation module generates control instructions according to the defect type and severity, and sends them to the laser welding robot adaptation module through the communication module.

[0074] As Figure 6 shown, Figure 6 Figure 1 is a schematic diagram of a laser welding robot adaptation module shown in an embodiment of the present invention. In Figure 6 this system, the laser welding robot adaptation module includes an interface conversion device and a parameter adjustment actuator. Among them, the interface conversion device can convert the digital signal of the control instruction into a signal format suitable for the laser welding robot control system. The parameter adjustment actuator adjusts the laser power, welding speed, or path of the laser welding robot according to the adapted instruction. As Figure 7 shown, Figure 7 Figure 2 is a schematic diagram of a parameter adjustment process shown in an embodiment of the present invention. In Figure 7 this system, after the control instruction generation module of the embedded control system generates a control instruction, it sends the control instruction to the interface conversion device in the laser welding robot adaptation module through the communication module for signal format conversion to obtain an adapted control signal and sends it to the parameter adjustment actuator in the laser welding robot adaptation module. The parameter adjustment actuator adjusts the laser power, welding speed, or path of the laser welding robot according to the adapted control signal.

[0075] In one embodiment, the embedded control system can also transmit the detection data and analysis results to the host computer in real time. The operator can grasp the welding quality situation in real time through the monitoring and management software of the host computer. Among them, the host computer stores and analyzes the historical data to provide data support for subsequent welding process optimization. At the same time, the operator can send control instructions to the embedded control system through the host computer to manually intervene in the welding process. During the welding process, the system continuously performs data acquisition, analysis, and adjustment to form a closed-loop control until the welding is completed. After the welding is completed, the system stops running and saves the relevant data of this welding.

[0076] As Figure 8 shown, Figure 8This is a schematic diagram of signal output between a host computer and an embedded control system shown in an embodiment of the present invention. In Figure 8 it, digital signals are transmitted between the communication module of the embedded control system and the host computer.

[0077] In one embodiment, an online detection system is designed, which at least includes: a high-precision vision sensor component, an infrared thermal imager component, a host computer, an embedded control system, a laser welding robot adaptation module, and a laser welding robot. It can monitor the welding quality in real time during the laser welding process. Even under complex welding conditions, it can accurately identify welding defects (such as pores, cracks, etc.), thereby improving the stability and consistency of welding quality. Through AI algorithms and real-time monitoring mechanisms, the parameter adjustment during the welding process is optimized to ensure that laser welding can complete the welding task efficiently and with high quality under diverse conditions such as different welding materials and welding speeds. As Figure 9 shown, Figure 9 This is a schematic diagram of an online welding quality detection method based on laser welding shown in an embodiment of the present invention. In Figure 9 it, during the laser welding process, the high-precision vision sensor component and the infrared thermal imager component can collect multi-source data for the welding area and send the multi-source data to the embedded control system. The embedded control system analyzes the multi-source data to obtain the welding quality detection result. In the case where the welding quality detection result is abnormal, a control instruction is sent to the laser welding robot adaptation module to control the laser welding robot to adjust the welding parameters. The laser welding robot adaptation module adjusts the laser power, welding speed, or path of the laser welding robot based on the control instruction, and controls the laser welding robot to perform subsequent welding for the welding area according to the adjusted laser power, welding speed, or path. The embedded control system can also transmit the detection data and analysis results to the host computer in real time, and the operator can grasp the welding quality situation in real time through the monitoring and management software of the host computer.

[0078] Combined with the above embodiments, in one embodiment, an online welding quality detection method based on laser welding is proposed, aiming to solve problems such as low efficiency, inability to correct errors in a timely manner, and increased production costs caused by the existing dependence on manual sampling inspection or offline detection for welding quality detection. This embodiment combines multi-sensor fusion with AI algorithms to realize real-time diagnosis and closed-loop control of welding quality, significantly improving the efficiency and accuracy of welding quality detection. As Figure 10 shown, Figure 10 This is a schematic diagram of an online welding quality detection method based on laser welding shown in an embodiment of the present invention. In Figure 10 it, the method can be executed according to the following steps: Step S1: Start the system: Turn on the laser welding robot and the on-line welding quality detection system to make each device enter the working state.

[0079] Step S2: The sensor collects data: The high-precision vision sensor collects data on the molten pool morphology, weld width and depth, and the infrared thermal imager collects data on the temperature distribution in the welding area.

[0080] Step S3: Transmit the data to the embedded control system: Transmit the collected multi-source data to the embedded control system through the data transmission line.

[0081] Step S4: Data preprocessing: The data processing module of the embedded control system performs preprocessing operations on the data, such as denoising and grayscale transformation.

[0082] Step S5: The AI algorithm analyzes the data: The AI analysis module uses the convolutional neural network algorithm (such as the pre-trained welding quality detection model) to analyze the preprocessed data to determine whether there are welding defects.

[0083] Step S6: Whether there are welding defects: Judge whether there are welding defects according to the AI analysis result.

[0084] Step S7: Generate control instructions: If there are welding defects, the control instruction generation module generates control instructions.

[0085] Step S8: Adjust the welding parameters: The laser welding robot adaptation module adjusts the laser power, welding speed or path according to the control instructions.

[0086] Step S9: Continue normal welding: If no welding defects are detected, the laser welding robot continues to perform normal welding according to the original parameters.

[0087] Step S10: Continue welding and monitoring: Continuously perform data collection, analysis and adjustment during the welding process.

[0088] Step S11: Whether the welding is completed: Judge whether the welding is completed.

[0089] Step S12: End: After the welding is completed, the system stops running.

[0090] Through the on-line welding quality detection method provided by this embodiment, there are at least the following advantages: 1. Improve accuracy: Multi-sensor fusion for data acquisition: A high-precision vision sensor and an infrared thermal imager are integrated at the end of the laser welding robot to synchronously acquire multi-source data. By using the multi-sensor fusion method of a high-precision vision sensor and an infrared thermal imager, the vision sensor can accurately capture the geometric features of the weld seam, while the infrared thermal imager can accurately monitor the temperature changes in the welding area. The two types of data complement each other, enabling the acquisition of multi-dimensional information during the welding process, avoiding the limitations of single-sensor detection, and greatly improving the accuracy of welding quality detection.

[0091] AI algorithm analysis: An AI algorithm (such as a convolutional neural network algorithm) is introduced to deeply analyze the multi-source data, and a correlation model between welding quality and these data is established. The AI algorithm can learn a large amount of welding data and the corresponding welding quality results, thereby accurately identifying the types and locations of welding defects (such as pores, cracks, etc.).

[0092] 2. Enhance safety: Real-time monitoring and closed-loop control: The system can monitor the quality of the welding process in real time. Once abnormal welding quality is detected, the embedded control system will immediately automatically adjust the laser power, welding speed, or path of the laser welding robot to form a closed-loop control.

[0093] Improved protection mechanism: Through real-time monitoring of the welding process, the system can promptly detect abnormal situations such as overvoltage and undervoltage and take corresponding protection measures. Among them, in an optional implementation, by analyzing the collected visual images and data, abnormal situations such as overvoltage and undervoltage are indirectly inferred. For example, the AI algorithm indirectly identifies voltage fluctuations through abnormalities in the molten pool morphology and temperature data (such as overvoltage may cause too high laser power, leading to excessive evaporation or spatter of the molten pool, and the vision sensor can detect an increase in the distortion of the molten pool morphology; undervoltage may cause insufficient laser power and abnormal temperature distribution in the molten pool (such as the temperature being lower than the threshold), and the abnormal temperature area is captured by the infrared thermal imager). In another optional implementation, it is also possible to take measures such as real-time monitoring of voltage through a voltage sensor built into the embedded control system or real-time interaction with the control system of the laser welding robot to promptly detect abnormal situations such as overvoltage and undervoltage.

[0094] 3. Improve reliability: Reduce the interference of human factors: Traditional manual sampling inspection or offline detection methods are easily affected by human factors, while the online detection system of this patent realizes automated and intelligent detection, reduces the influence of human factors, and improves the reliability and consistency of the detection results.

[0095] Data storage and analysis: The host computer stores and analyzes the detection data and analysis results in real time, providing strong data support for subsequent welding process optimization. By analyzing historical data, potential problems and patterns in the welding process can be discovered, and welding process parameters can be adjusted in a timely manner to further improve the reliability of welding quality.

[0096] 4. Improve real-time performance: Real-time data acquisition and processing: High-precision vision sensors and infrared thermal imagers can collect data during the welding process in real time and transmit the data to the embedded control system quickly through high-speed data transmission lines. The embedded control system processes and analyzes the data in real time and can judge whether there are abnormalities in welding quality within a short time.

[0097] Real-time adjustment of welding parameters: When welding quality abnormalities are detected, the embedded control system can generate control instructions in real time and adjust the parameters of the laser welding robot in real time through the laser welding robot adaptation module to ensure stable and reliable welding quality.

[0098] 5. Save costs: Reduce manual inspection costs: Traditional manual sampling inspection or off-line inspection requires a large amount of manpower and time, with high costs. The on-line inspection system of this patent realizes automated inspection, reduces the workload of manual inspection, and lowers the labor cost. At the same time, the system can discover welding defects in real time and make adjustments, reducing the product scrap and rework costs caused by welding defects.

[0099] Improve production efficiency: Real-time monitoring and closed-loop control mechanisms make the welding process more stable and efficient, reducing production downtime and adjustment time caused by welding quality problems.

[0100] 6. Strong compatibility and adaptability: Multi-scenario application: The multi-sensor fusion and AI algorithm technology adopted by the system has strong compatibility and adaptability and can be applied to different types of welding as well as different welding materials and processes.

[0101] Scalability: The architecture design of the system has good scalability, and new sensors can be added or the AI algorithm can be optimized conveniently to adapt to the ever-developing welding technology and quality inspection requirements.

[0102] It should be noted that for method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequences, because according to the embodiments of the present invention, certain steps can be carried out in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0103] Based on the same inventive concept, an embodiment of the present invention provides a welding quality detection system. Referring to Figure 11 , Figure 11 is a structural block diagram of a welding quality detection system provided by an embodiment of the present invention. As shown in Figure 11 , the system at least includes: different types of sensors and an embedded control system; The different types of sensors are used to collect multi-source data for the welding area during the welding process and send the multi-source data to the embedded control system; the multi-source data includes: visual images and temperature distribution data; The embedded control system is used to obtain the multi-source data, input the multi-source data into a pre-trained welding quality detection model to obtain probabilities of multiple welding defect categories; in the case that the maximum probability exceeds the probability threshold, determine that the welding quality detection result of the welding area is abnormal welding quality, and use the welding defect category corresponding to the maximum probability as the defect type of the welding area; determine the defect position corresponding to the defect type through bounding box regression or feature map coordinate mapping; Wherein, the pre-trained welding quality detection model is trained based on the sample multi-source data collected for the sample welding area during the historical welding process and the true defect information of the sample welding area, and the true defect information is the defect type and defect position manually marked for the sample welding area.

[0104] Optionally, the system further includes: a welding robot adaptation module and a welding robot; The embedded control system is further used to generate a control instruction and send the control instruction to the welding robot adaptation module in the case that the welding quality detection result is abnormal welding quality; The welding robot adaptation module is used to adjust the welding parameters based on the control instruction and send the adjusted welding parameters to the welding robot; The welding robot is used to weld the welding area according to the adjusted welding parameters.

[0105] Optionally, the embedded control system is specifically used for: Analyze the visual image to obtain molten pool morphology information, weld width information, and weld depth information; Convert the temperature distribution data to obtain a heat map; Stitch the visual image carrying the molten pool morphology information, the weld width information, and the weld depth information with the heat map representing the temperature distribution data to obtain a stitched image; Input the spliced image into the pre-trained welding quality detection model to obtain the probabilities of the multiple welding defect categories.

[0106] Optionally, the embedded control system is specifically configured to: Project a light source onto the welding area and obtain a visual image collected for the welding area; the molten pool morphology information at least includes: the concave-convex information, depth information, width information, and surface physical information of the molten pool; Analyze the visual image to determine the distortion information of the reflected light generated by projecting the light source onto the welding area, where the distortion information of the reflected light at least includes: geometric distortion information and gray value distortion information; Based on the geometric distortion information, determine the three-dimensional morphology information of the molten pool; Based on the three-dimensional morphology information, determine the weld depth information, the concave-convex information, and depth information of the molten pool; Based on the gray value distortion information, determine the surface physical information and width information of the molten pool.

[0107] Optionally, the pre-trained welding quality detection model at least includes: multiple convolutional layers, pooling layers, and fully connected layers; the embedded control system is specifically configured to: Extract features of different dimensions from the spliced image through the multiple convolutional layers to obtain a first image feature; Fuse the temperature distribution data, the molten pool morphology information, the weld width information, and the weld depth information with the first image feature to obtain a second image feature; Perform dimensionality reduction processing on the second image feature through the pooling layer to obtain welding data features; Process the welding data features through the fully connected layer to obtain the probabilities of the multiple welding defect categories.

[0108] Optionally, the embedded control system is specifically configured to: Map the welding data features to different welding defect categories through the fully connected layer to obtain the probabilities of the multiple welding defect categories.

[0109] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the welding quality detection method as described in any one of the above embodiments of the present invention.

[0110] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, as Figure 12 shown. Figure 12It is a schematic diagram of an electronic device shown in an embodiment of the present invention. The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes, it implements the steps in the welding quality detection method described in any of the above embodiments of the present invention.

[0111] For the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment.

[0112] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other.

[0113] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0114] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0115] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide for implementing the process Figure 1 one process or multiple processes and / or blocks Figure 1 steps for the functions specified in one block or multiple blocks.

[0117] Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

[0118] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0119] The above has introduced in detail a welding quality detection method, device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A welding quality detection method, characterized in that: The method comprises: Acquire multi-source data collected for the welding area during the welding process, wherein the multi-source data includes data collected by different types of sensors; the multi-source data includes: visual images and temperature distribution data; Inputting the multi-source data into a pre-trained welding quality detection model to obtain the probabilities of multiple welding defect categories; In the case where the maximum probability exceeds the probability threshold, determining that the welding quality detection result of the welding area is abnormal welding quality, and taking the welding defect category corresponding to the maximum probability as the defect type of the welding area; Determine the defect position corresponding to the defect type by bounding box regression or feature map coordinate mapping; Among them, the pre-trained welding quality inspection model is trained based on sample multi-source data collected from sample welding areas during historical welding processes and real defect information of sample welding areas. The real defect information is the defect type and defect location manually marked for the sample welding area.

2. The welding quality detection method according to claim 1, characterized in that: The method further comprises: When the welding quality detection result shows that the welding quality is abnormal, the welding parameters are adjusted, and the welding area is welded according to the adjusted welding parameters.

3. The welding quality detection method according to claim 1 or 2, characterized in that: The multi-source data is input into a pre-trained welding quality inspection model to obtain the probabilities of multiple welding defect categories, including: Analyzing the visual image to obtain molten pool morphology information, weld width information, and weld depth information; Converting the temperature distribution data to obtain a thermal map; Splicing the visual image carrying the molten pool morphology information, the weld width information and the weld depth information with the thermal map representing the temperature distribution data to obtain a spliced ​​image; The stitched image is input into the pre-trained welding quality detection model to obtain the probabilities of the multiple welding defect categories.

4. The welding quality detection method according to claim 3, characterized in that: Acquire multi-source data collected on the weld area during the welding process, including: Projecting a light source onto the welding area to obtain a visual image collected for the welding area; The molten pool morphology information at least includes: convex-concave information, depth information, width information and surface physical information of the molten pool; analyzing the visual image to obtain the molten pool morphology information and weld depth information includes: Analyzing the visual image to determine distortion information of reflected light generated by the light source projected onto the welding area, wherein the distortion information of the reflected light at least includes: geometric distortion information and gray value distortion information; Determining three-dimensional morphology information of the molten pool based on the geometric distortion information; Based on the three-dimensional morphology information, determine the weld depth information, the concave-convex information and the depth information of the molten pool; Based on the grayscale value distortion information, the surface physical information and width information of the molten pool are determined.

5. The welding quality detection method according to claim 3, characterized in that: The pre-trained welding quality detection model at least includes: multiple convolutional layers, pooling layers and fully connected layers; the spliced ​​image is input into the pre-trained welding quality detection model to obtain the probabilities of the multiple welding defect categories, including: Extracting features of different dimensions from the spliced ​​image through the multi-layer convolutional layer to obtain a first image feature; The temperature distribution data, the molten pool morphology information, the weld width information and the weld depth information are integrated with the first image feature to obtain a second image feature; Performing dimensionality reduction processing on the second image feature through the pooling layer to obtain welding data features; The welding data features are processed by the fully connected layer to obtain the probabilities of the multiple welding defect categories.

6. The welding quality detection method according to claim 5, characterized in that: Processing the welding data features through the fully connected layer to obtain the probabilities of the multiple welding defect categories includes: The welding data features are mapped to different welding defect categories through the fully connected layer to obtain the probabilities of the multiple welding defect categories.

7. A welding quality detection system, characterized in that: The system at least includes: different types of sensors and an embedded control system; The different types of sensors are used to collect multi-source data for the welding area during the welding process and send the multi-source data to the embedded control system; the multi-source data includes: visual images and temperature distribution data; The embedded control system is used to acquire the multi-source data and input the multi-source data into a pre-trained welding quality detection model to obtain the probabilities of multiple welding defect categories; when the maximum probability exceeds the probability threshold, determine that the welding quality detection result of the welding area is abnormal welding quality, and use the welding defect category corresponding to the maximum probability as the defect type of the welding area; determine the defect position corresponding to the defect type through bounding box regression or feature map coordinate mapping; Among them, the pre-trained welding quality inspection model is trained based on sample multi-source data collected from sample welding areas during historical welding processes and real defect information of sample welding areas. The real defect information is the defect type and defect location manually marked for the sample welding area.

8. The welding quality inspection system according to claim 7, characterized in that: The system further comprises: a welding robot adaptation module and a welding robot; The embedded control system is further used to generate a control instruction when the welding quality detection result is abnormal welding quality, and send the control instruction to the welding robot adapter module; The welding robot adaptation module is used to adjust the welding parameters based on the control instructions and send the adjusted welding parameters to the welding robot; The welding robot is used to weld the welding area according to the adjusted welding parameters.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by the processor, the welding quality detection method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the welding quality detection method according to any one of claims 1 to 6 is implemented.

Images