Method, system and equipment for identifying welding defects of automobile parts and medium

Abstract

The invention provides a welding defect identification method, system and equipment for an automobile part and a medium, and relates to the technical field of automobile welding.The method comprises the steps that a structured light vision sensor and a high-frequency ultrasonic sensor are used for obtaining three-dimensional structured light data of a to-be-welded area of the automobile part and sound wave signals generated in the welding process; obtaining characteristic frequency information used for identifying welding internal defects; the method comprises the following steps: capturing a heat distribution image of a to-be-welded area of an automobile part through a thermal imaging camera to obtain area feature information used for identifying welding defect thermal anomaly, and obtaining magneto-optical signal feature information used for identifying welding defects; performing defect identification and classification on the fused feature representation by training a deep learning model to obtain a defect identification result; and welding parameters are adjusted through a simulated annealing algorithm, so that the welding defects of the automobile parts are recognized. According to the invention, various welding defects can be identified, so that the accuracy and efficiency of welding quality detection are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile welding, and in particular to a method, system, equipment and medium for identifying welding defects of automobile parts. Background Art

[0002] With the rapid development of industrial automation, welding technology plays a vital role in the automotive manufacturing industry. Traditional welding quality inspection methods rely on manual inspection or simple automated equipment, which have the problems of slow inspection speed, low accuracy and low efficiency. In order to improve the inspection efficiency and accuracy of welding quality, existing technologies try to use machine vision and ultrasonic inspection.

[0003] However, these technologies can often only detect defects on the weld surface, but have difficulty identifying defects inside the weld and have poor adaptability in complex environments. Summary of the invention

[0004] The purpose of the present invention is to provide a method, system, device and medium for identifying welding defects of automobile parts to improve the above problems. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows: In a first aspect, the present application provides a method for identifying welding defects of automotive parts, comprising: The three-dimensional structured light data of the area to be welded of the automobile parts and the acoustic wave signals generated during the welding process are obtained by using structured light vision sensors and high-frequency ultrasonic sensors, and the three-dimensional structured light data and the acoustic wave signals are preprocessed respectively, including filtering and denoising the three-dimensional structured light data to obtain edge and contour information for identifying the geometric features of the weld, and performing spectrum analysis on the acoustic wave signals to remove background noise and obtain characteristic frequency information for identifying internal defects of the weld; The thermal distribution image of the area to be welded of the automotive parts is captured by a thermal imaging camera, and the magnetic field change in the welding area is detected by magneto-optical imaging technology. The temperature of the thermal distribution image is calibrated to obtain the regional characteristic information for identifying thermal anomalies of welding defects. The magneto-optical imaging data is spatially filtered to obtain the magneto-optical signal characteristic information for identifying welding defects. Based on the edge and contour information, characteristic frequency information, regional characteristic information and magneto-optical signal characteristic information, a fused feature representation is obtained through deep learning algorithm processing. A deep learning model is trained to perform defect recognition and classification on the fused feature representation to obtain defect recognition results. The model can automatically learn the characteristics of welding defects and classify them into different categories, including cracks, pores and incomplete penetration. Based on the defect recognition results, the welding quality is evaluated, and the welding parameters are adjusted through the simulated annealing algorithm to generate a welding quality report, thereby identifying the welding defects of automotive parts.

[0005] Preferably, the structured light vision sensor and the high-frequency ultrasonic sensor are used to obtain the three-dimensional structured light data of the area to be welded of the automobile parts and the acoustic wave signal generated during the welding process, and the three-dimensional structured light data and the acoustic wave signal are preprocessed respectively, including filtering and denoising the three-dimensional structured light data to obtain edge and contour information for identifying the geometric features of the weld, and performing spectrum analysis on the acoustic wave signal to remove background noise to obtain characteristic frequency information for identifying internal defects of the weld, including: The original three-dimensional structured light data of the area to be welded of the automobile parts collected by the structured light vision sensor is processed by Fourier transform to obtain the frequency domain representation of the weld; The frequency domain representation of the weld is processed using inverse Fourier transform and Gaussian filtering to obtain enhanced weld edge and contour information; The original sound wave signal generated during the welding process is collected by a high-frequency ultrasonic sensor and processed by short-time fractional Fourier transform to obtain a time-frequency domain representation, which includes analyzing the local spectrum characteristics of the sound wave signal by short-time fractional Fourier transform to provide time-frequency characteristics for identifying internal defects in welding; The time-frequency domain representation is processed by wavelet denoising to obtain the characteristic frequency information used to identify internal welding defects.

[0006] Preferably, the thermal distribution image of the area to be welded of the automobile parts is captured by a thermal imaging camera, and the magnetic field change of the welding area is detected by magneto-optical imaging technology, and the temperature of the thermal distribution image is calibrated to obtain regional characteristic information for identifying thermal anomalies of welding defects, and the magneto-optical imaging data is spatially filtered to obtain magneto-optical signal characteristic information for identifying welding defects, which includes: Based on the thermal distribution image of the area to be welded of the automobile parts captured by the thermal imaging camera, the regional characteristic information for identifying thermal anomalies of welding defects is obtained through calculations based on the blackbody radiation law and the Stefan-Boltzmann law. Based on the thermal distribution image, after 3D reconstruction processing using binocular vision technology, a 3D temperature distribution model for identifying welding defects is obtained using a pinhole camera model and a stereo matching algorithm. The stereo matching algorithm includes feature extraction, feature matching, and parallax calculation, and ultimately the three-dimensional coordinates of each pixel are obtained. Based on the 3D temperature distribution model, the thermal characteristics and defect locations of the welding area are obtained through the belief propagation algorithm, which optimizes the parameter estimation of the model through an iterative process; After the thermal characteristics and defect positions are processed by the spatial filter and the coherent optical image processing system, the enhanced magneto-optical signal characteristic information for identifying welding defects is obtained.

[0007] Preferably, the edge and contour information, characteristic frequency information, regional characteristic information and magneto-optical signal characteristic information are processed by a deep learning algorithm to obtain a fused feature representation, and a deep learning model is trained to perform defect recognition and classification on the fused feature representation to obtain a defect recognition result, which includes: According to the structured light feature vector, the ultrasonic feature vector, the thermal imaging feature vector and the magneto-optical feature vector, after being processed by the feature fusion layer, a fused feature representation is obtained; The fused feature representation is trained using a deep learning model, wherein the training process uses a cross entropy loss function and a back propagation algorithm to optimize the model parameters to obtain the trained model parameters; The trained model parameters are calculated using the forward propagation method, and the model outputs the predicted probability of each defect category. Finally, the defect type is determined based on the category with the highest probability, and the specific location of the defect is determined by combining image segmentation technology to obtain the defect recognition result.

[0008] Preferably, the welding quality is evaluated based on the defect identification result, and the welding parameters are adjusted by a simulated annealing algorithm to generate a welding quality report, thereby identifying the welding defects of automobile parts, including: Based on the defect type, quantity and location in the defect identification results, combined with the welding process standards, the comprehensive score is calculated using defect density, defect distribution and defect severity indicators; The comprehensive scores were processed using simulated annealing algorithm to obtain optimized welding parameters; The optimized welding parameters are processed through a feedback control mechanism, which includes monitoring the welding process and adjusting the welding equipment parameters in real time to obtain the final adjusted welding parameters, thereby identifying welding defects of automotive parts.

[0009] In a second aspect, the present application also provides a welding defect recognition system for automotive parts, comprising: Processing module: used to obtain the three-dimensional structured light data of the area to be welded of the automobile parts and the acoustic wave signal generated during the welding process by using the structured light vision sensor and the high-frequency ultrasonic sensor, and pre-process the three-dimensional structured light data and the acoustic wave signal respectively, including filtering and denoising the three-dimensional structured light data to obtain the edge and contour information for identifying the geometric features of the weld, and performing spectrum analysis on the acoustic wave signal to remove the background noise and obtain the characteristic frequency information for identifying the internal defects of the weld; Calibration module: used to capture the thermal distribution image of the area to be welded of the automotive parts through a thermal imaging camera, and use magneto-optical imaging technology to detect the magnetic field change in the welding area, and perform temperature calibration on the thermal distribution image to obtain regional feature information for identifying thermal anomalies of welding defects, and perform spatial filtering on the magneto-optical imaging data to obtain magneto-optical signal feature information for identifying welding defects; Training module: used to obtain fused feature representations based on edge and contour information, characteristic frequency information, regional feature information and magneto-optical signal feature information through deep learning algorithm processing, and to obtain defect recognition results by training a deep learning model to perform defect recognition and classification on the fused feature representations. The model can automatically learn the characteristics of welding defects and classify them into different categories, including cracks, pores and incomplete penetration. Identification module: used to evaluate welding quality based on defect identification results, adjust welding parameters through simulated annealing algorithm, generate welding quality report, and thus identify welding defects of automotive parts.

[0010] In a third aspect, the present application also provides a welding defect identification device for automobile parts, comprising: Memory for storing computer programs; A processor is used to implement the steps of the method for identifying welding defects of automobile parts when executing the computer program.

[0011] In a fourth aspect, the present application further provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned welding defect identification method based on automobile parts are implemented.

[0012] The beneficial effects of the present invention are: The present invention realizes comprehensive detection of welding area by combining structured light vision sensor, high-frequency ultrasonic sensor, thermal imaging camera and magneto-optical imaging technology. First, the three-dimensional structured light data of welding area is obtained by structured light vision sensor, and then the clear edge and contour information of weld are extracted by Fourier transform and Gaussian filtering. Then, the acoustic wave signal generated in welding process is collected by high-frequency ultrasonic sensor, and the characteristic frequency information of welding defect is extracted by short-time Fourier transform and background noise removal technology. In addition, the thermal distribution image of welding area is captured by thermal imaging camera, and temperature calibration is adopted to obtain the characteristic information of thermal abnormal area of ​​welding defect. Finally, the magnetic field change of welding area is detected by magneto-optical imaging technology, and spatial filtering is performed to obtain the characteristic information of magneto-optical signal of welding defect. These characteristic information are input into a deep learning model, and a model capable of automatically identifying and classifying welding defects is obtained by training. The model can identify a variety of welding defects including cracks, pores and incomplete penetration, thereby significantly improving the accuracy and efficiency of welding quality detection.

[0013] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by implementing the embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 It is a schematic flow chart of a welding defect identification method for automobile parts according to an embodiment of the present invention; Figure 2 It is a structural schematic diagram of a welding defect recognition system for automobile parts according to an embodiment of the present invention; Figure 3 It is a schematic diagram of the structure of the welding defect identification equipment for automobile parts described in an embodiment of the present invention.

[0016] In the figure: 701, processing module; 702, calibration module; 703, training module; 704, recognition module; 800, welding defect recognition equipment for automobile parts; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0018] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. Example 1

[0019] This embodiment provides a method for identifying welding defects of automobile parts.

[0020] See also Figure 1 , the figure shows that the method includes step S100, step S200, step S300 and step S400.

[0021] S100, using a structured light vision sensor and a high-frequency ultrasonic sensor to obtain three-dimensional structured light data of the area to be welded of the automobile parts and the acoustic wave signal generated during the welding process, and preprocessing the three-dimensional structured light data and the acoustic wave signal respectively, including filtering and denoising the three-dimensional structured light data to obtain edge and contour information for identifying geometric features of the weld, performing spectrum analysis on the acoustic wave signal, removing background noise, and obtaining characteristic frequency information for identifying internal defects in the weld.

[0022] It can be understood that the step S100 includes S101, S102, S103 and S104, wherein: S101, obtaining a frequency domain representation of the weld by Fourier transforming the original three-dimensional structured light data of the area to be welded of the automobile parts collected by the structured light vision sensor; It should be noted that this step utilizes the edge enhancement feature of Fourier transform in image processing to highlight the geometric features of the weld.

[0023] S102, processing the frequency domain representation of the weld by inverse Fourier transform and Gaussian filtering to obtain enhanced weld edge and contour information; It should be noted that this step uses Gaussian filter smoothing to effectively remove noise while retaining the geometric features of the weld.

[0024] S103, the original sound wave signal generated during the welding process collected by the high-frequency ultrasonic sensor is processed by short-time fractional Fourier transform to obtain a time-frequency domain representation, including analyzing the local spectrum characteristics of the sound wave signal by short-time fractional Fourier transform to provide time-frequency characteristics for identifying internal defects in welding; S104, processing the time-frequency domain representation through wavelet denoising to obtain characteristic frequency information for identifying internal welding defects.

[0025] It should be noted that this step utilizes the multi-resolution analysis capability of wavelet transform to effectively remove background noise and extract characteristic frequency components related to welding defects.

[0026] S200. Capture the thermal distribution image of the area to be welded of the automobile parts by means of a thermal imaging camera, detect the magnetic field change in the welding area by means of magneto-optical imaging technology, perform temperature calibration on the thermal distribution image respectively, obtain regional characteristic information for identifying thermal anomalies of welding defects, perform spatial filtering on the magneto-optical imaging data, and obtain magneto-optical signal characteristic information for identifying welding defects.

[0027] In this step, the multimodal data obtained by structured light vision sensors, high-frequency ultrasonic sensors, thermal imaging cameras and magneto-optical imaging technology can be accurately extracted through Fourier transform, short-time Fourier transform, temperature calibration and spatial filtering, etc., to accurately extract the geometry, frequency, thermal anomaly and magneto-optical signal characteristics of the welding area. These characteristic information provides a rich data basis for welding defect identification. Compared with the traditional single data source method, the present invention can more comprehensively capture the characteristics of welding defects and improve the accuracy and reliability of identification.

[0028] It can be understood that step S200 includes S201, S202, S203 and S204, wherein: S201. Based on the thermal distribution image of the area to be welded of the automobile parts captured by the thermal imaging camera, the regional characteristic information for identifying thermal anomalies of welding defects is obtained by calculation using the blackbody radiation law and the Stefan-Boltzmann law. The calculation formula of the blackbody radiation law is as follows: In the formula, j ∗ is the total power (radiance) radiated per unit area of ​​a black body surface, T is the thermodynamic temperature of the black body itself, σ is the Stefan–Boltzmann constant, ϵ is the radiation coefficient of the black body; S202, based on the thermal distribution image, after 3D reconstruction processing using binocular vision technology, using a pinhole camera model and a stereo matching algorithm, a 3D temperature distribution model for identifying welding defects is obtained, wherein the stereo matching algorithm includes feature extraction, feature matching and parallax calculation, and finally the three-dimensional coordinates of each pixel point are obtained; It should be noted that the pinhole camera model is a mathematical model based on projective geometry, which maps points in the three-dimensional world onto a two-dimensional image plane.

[0029] S203, according to the 3D temperature distribution model, a belief propagation algorithm is used for processing, and the algorithm optimizes the parameter estimation of the model through an iterative process to obtain the thermal characteristics and defect locations of the welding area; It should be noted that the belief propagation algorithm is an algorithm based on graph theory. It achieves global optimization by locally optimizing each node and is often used to solve the disparity estimation problem in stereo vision.

[0030] S204, the thermal characteristics and defect positions are processed by a spatial filter and a coherent optical image processing system to obtain enhanced magneto-optical signal characteristic information for identifying welding defects.

[0031] It should be noted that spatial filters are usually used to eliminate noise and irrelevant details in images, while coherent optical image processing systems use the coherent properties of light to enhance the contrast and resolution of images, thereby improving the accuracy of defect detection.

[0032] S300, based on edge and contour information, characteristic frequency information, regional characteristic information and magneto-optical signal characteristic information, after being processed by a deep learning algorithm, a fused feature representation is obtained, and a deep learning model is trained to perform defect recognition and classification on the fused feature representation to obtain a defect recognition result, wherein the model can automatically learn the characteristics of welding defects and classify them into different categories, including cracks, pores and incomplete penetration.

[0033] It can be understood that in step S300, S301, S302 and S303, wherein: S301, according to the structured light feature vector, the ultrasonic feature vector, the thermal imaging feature vector and the magneto-optical feature vector, through the feature fusion layer processing, obtain a fused feature representation; It should be noted that feature fusion layer processing includes but is not limited to fusion strategies in deep learning, such as weighted summation, feature concatenation, or depth-wise separable convolution.

[0034] S302, using a deep learning model to train the fused feature representation, wherein the training process uses a cross entropy loss function and a back propagation algorithm to optimize model parameters to obtain trained model parameters; It should be noted that the extracted feature vectors are fused through a specific fusion strategy. For example, the importance of different features can be combined by weighted summation, or different features can be connected into a longer feature vector by feature concatenation, or deep separable convolution can be used to further extract the deep representation of the fused features; the fused feature representation is input into the deep learning model for training. During the training process, the model learns how to identify different welding defects based on the fused feature representation. The cross entropy loss function and back propagation algorithm are usually used to optimize the model parameters during the training process; after the training is completed, the model is used to identify defects in new welding areas. The model outputs the predicted probability of each possible defect category through forward propagation calculation, and finally determines the defect type according to the category with the highest probability, and may combine image segmentation technology to determine the specific location of the defect. This process reflects the powerful ability of deep learning in multimodal data fusion and complex task processing, especially in the scene of welding defect recognition, which requires the integration of multiple information for accurate judgment.

[0035] S303. The trained model parameters are calculated by the forward propagation method. The model outputs the predicted probability of each defect category. Finally, the defect type is determined according to the category with the highest probability. The specific location of the defect is determined by combining image segmentation technology to obtain the defect recognition result.

[0036] It should be noted that in these steps, the feature extraction network adopts different network structures for different types of data. These network structures can be specially designed for specific types of data or general network structures, but they have been properly adjusted and trained to adapt to specific types of data. The feature fusion layer is the creation point, which determines how to effectively combine features from different modalities to improve the accuracy of defect recognition. In this embodiment, deep learning model training is a key link in the whole process, which ensures that the model can learn the ability to distinguish different welding defects from the fused feature representation; finally, the defect recognition and classification step is to apply the trained model to the actual welding area to identify and locate defects.

[0037] In this example, in the feature fusion stage, the present invention uses a deep learning algorithm to fuse multimodal features, a process that involves complex data processing and feature extraction techniques. For example, the key features are automatically learned and extracted through a convolutional neural network (CNN), and then the features of different modes are integrated through a feature fusion layer to obtain a comprehensive feature representation. This fusion method makes full use of the complementary information of each modal data and improves the performance of welding defect recognition; in the deep learning model training process, advanced optimization algorithms such as simulated annealing algorithms are used to optimize welding parameters. The simulated annealing algorithm simulates the physical annealing process and accepts new solutions with a certain probability, thereby avoiding falling into a local optimal solution and finding a global optimal solution or an approximate optimal solution. In welding parameter optimization, the algorithm dynamically adjusts welding parameters such as current, voltage, and welding speed according to the welding quality score to improve welding quality.

[0038] S400, based on the defect recognition results, evaluate the welding quality, adjust the welding parameters through the simulated annealing algorithm, generate a welding quality report, and thus identify the welding defects of automobile parts.

[0039] It can be understood that the step S400 includes S401, S402 and S403, wherein: S401. Calculate the comprehensive score based on the defect type, quantity and location in the defect identification results and the welding process standards using defect density, defect distribution and defect severity indicators; S402, using a simulated annealing algorithm to process the comprehensive score to obtain optimized welding parameters, the calculation formula of which is as follows: In the formula, Δ E is the energy difference between the new solution and the current solution, T is the current temperature, P (Δ E ) is the probability that the system accepts a new solution (even if its objective function value is worse than that of the current solution), and exp is a natural exponential function; S403, according to the optimized welding parameters, the feedback control mechanism is processed, wherein the feedback control mechanism includes monitoring the welding process and adjusting the welding equipment parameters in real time to obtain the final adjusted welding parameters, thereby identifying the welding defects of the automobile parts.

[0040] In this process, the specific processing methods of each step are as follows: Step 1: The welding quality assessment algorithm can use a rule-based system that scores the welding quality based on the defect type, quantity and location in the defect identification results combined with the welding process standards. For example, indicators such as defect density, defect distribution and defect severity can be used to calculate a comprehensive score.

[0041] Step 2: The simulated annealing algorithm is a heuristic search algorithm that simulates the particle motion in the physical annealing process to find the global optimal solution to the problem. In welding parameter optimization, the algorithm can adjust parameters such as current, voltage, and welding speed according to the welding quality score to improve the welding quality. The algorithm iteratively searches for the optimal parameters by randomly selecting new parameters and calculating their impact on the welding quality score, and then deciding whether to accept the new parameters based on a probability function.

[0042] Step 3: The report generation algorithm integrates the optimized welding parameters and welding quality scores into a detailed welding quality report. This report can include the adjustment history of welding parameters, the change in quality score after each adjustment, the statistics of defect identification, and improvement suggestions. The report can also contain charts and images to intuitively display key information of the welding process.

[0043] These three steps combine welding quality assessment, parameter optimization and report generation to provide a complete welding defect identification and quality control process. In this way, the quality and efficiency of the welding process can be ensured while providing valuable feedback information to the welding operator.

[0044] In this step, an automated report generation algorithm is used to generate a welding quality report. The algorithm includes not only the type and location of welding defects, but also the adjustment history and improvement suggestions of welding parameters. The generation of such a detailed report provides valuable feedback information for welding operators, which helps to understand the problems in the welding process and guide subsequent process improvements. Through the method of the present invention, real-time monitoring and evaluation of welding quality can be achieved, welding defects can be discovered and adjusted in time, thereby reducing the generation of defective products and improving production efficiency and product quality. The method can also be applied to automated welding systems to achieve intelligent control of the welding process and provide technical support for intelligent manufacturing. Example 2

[0045] like Figure 2 As shown, this embodiment provides a welding defect recognition system for automobile parts, see Figure 2 The system comprises: Processing module 701: used to obtain three-dimensional structured light data of the to-be-welded area of ​​the automobile parts and the acoustic wave signal generated during the welding process by using the structured light vision sensor and the high-frequency ultrasonic sensor, and pre-process the three-dimensional structured light data and the acoustic wave signal respectively, including filtering and denoising the three-dimensional structured light data to obtain edge and contour information for identifying the geometric features of the weld, and performing spectrum analysis on the acoustic wave signal to remove background noise and obtain characteristic frequency information for identifying internal defects of the weld; Calibration module 702: used to capture the thermal distribution image of the area to be welded of the automobile parts through a thermal imaging camera, and use magneto-optical imaging technology to detect the magnetic field change in the welding area, and perform temperature calibration on the thermal distribution image to obtain regional feature information for identifying thermal anomalies of welding defects, and perform spatial filtering on the magneto-optical imaging data to obtain magneto-optical signal feature information for identifying welding defects; Training module 703: used to obtain fused feature representations based on edge and contour information, feature frequency information, regional feature information and magneto-optical signal feature information through deep learning algorithm processing, and to perform defect recognition and classification on the fused feature representations by training a deep learning model to obtain defect recognition results, wherein the model can automatically learn the features of welding defects and classify them into different categories, including cracks, pores and incomplete penetration; Identification module 704: used to evaluate welding quality based on defect identification results, adjust welding parameters through simulated annealing algorithm, generate welding quality report, and thus identify welding defects of automobile parts.

[0046] Specifically, the processing module 701 includes: The first processing unit is used to obtain the frequency domain representation of the weld according to the original three-dimensional structured light data of the area to be welded of the automobile parts collected by the structured light vision sensor through Fourier transform processing; The second processing unit is used to process the frequency domain representation of the weld by using inverse Fourier transform and Gaussian filtering to obtain enhanced weld edge and contour information; The third processing unit is used to obtain the time-frequency domain representation of the original sound wave signal generated during the welding process based on the acquisition of the high-frequency ultrasonic sensor through short-time fractional Fourier transform, including analyzing the local spectrum characteristics of the sound wave signal through short-time fractional Fourier transform to provide time-frequency characteristics for identifying internal defects in welding; The fourth processing unit is used to process the time-frequency domain representation through wavelet denoising to obtain characteristic frequency information for identifying internal defects in welding.

[0047] Specifically, the calibration module 702 includes: Calculation unit: used to obtain regional characteristic information for identifying thermal anomalies of welding defects based on the thermal distribution image of the area to be welded of the automobile parts captured by the thermal imaging camera, through the blackbody radiation law and the Stefan-Boltzmann law. The calculation formula of the blackbody radiation law is as follows: In the formula, j ∗ is the total power (radiance) radiated per unit area of ​​a black body surface, T is the thermodynamic temperature of the black body itself, σ is the Stefan–Boltzmann constant, ϵ is the radiation coefficient of the black body; The fifth processing unit is used to obtain a 3D temperature distribution model for identifying welding defects based on the thermal distribution image after 3D reconstruction processing using binocular vision technology, using a pinhole camera model and a stereo matching algorithm, wherein the stereo matching algorithm includes feature extraction, feature matching, and parallax calculation, and finally obtains the three-dimensional coordinates of each pixel point; Estimation unit: used to obtain the thermal characteristics and defect locations of the welding area based on the 3D temperature distribution model after being processed by the belief propagation algorithm, which optimizes the parameter estimation of the model through an iterative process; The sixth processing unit is used to process the thermal characteristics and defect positions through a spatial filter and a coherent optical image processing system to obtain enhanced magneto-optical signal characteristic information for identifying welding defects.

[0048] Specifically, the training module 703 includes: Fusion unit: used to obtain fused feature representation according to the structured light feature vector, ultrasonic feature vector, thermal imaging feature vector and magneto-optical feature vector after being processed by the feature fusion layer; Training unit: used to train the fused feature representation using a deep learning model. The training process uses a cross entropy loss function and a back propagation algorithm to optimize the model parameters and obtain the trained model parameters. Determination unit: used to calculate the trained model parameters through the forward propagation method. The model outputs the predicted probability of each defect category. Finally, the defect type is determined according to the category with the highest probability, and the specific location of the defect is determined by combining image segmentation technology to obtain the defect recognition result.

[0049] Specifically, the identification module 704 includes: Comprehensive score calculation unit: used to calculate the comprehensive score based on the defect type, quantity and location in the defect identification results, combined with the welding process standards, using defect density, defect distribution and defect severity indicators; Optimization unit: used to process the comprehensive score using the simulated annealing algorithm to obtain optimized welding parameters. The calculation formula is as follows: In the formula, Δ E is the energy difference between the new solution and the current solution, T is the current temperature, P (Δ E ) is the probability that the system accepts a new solution (even if its objective function value is worse than that of the current solution), and exp is a natural exponential function; Adjustment unit: used to process the optimized welding parameters through a feedback control mechanism, wherein the feedback control mechanism includes monitoring the welding process and adjusting the welding equipment parameters in real time to obtain the final adjusted welding parameters, thereby identifying welding defects of automotive parts.

[0050] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here. Example 3

[0051] Corresponding to the above method embodiment, this embodiment also provides a welding defect recognition device for automobile parts. The welding defect recognition device for automobile parts described below and the welding defect recognition method for automobile parts described above can refer to each other.

[0052] Figure 3 FIG. 8 is a block diagram of a welding defect recognition device 800 for automobile parts according to an exemplary embodiment. Figure 3 As shown, the welding defect recognition device 800 for automobile parts includes: a processor 801 and a memory 802. The welding defect recognition device 800 for automobile parts also includes one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0053] The processor 801 is used to control the overall operation of the welding defect recognition device 800 for automobile parts to complete all or part of the steps in the above-mentioned welding defect recognition method for automobile parts. The memory 802 is used to store various types of data to support the operation of the welding defect recognition device 800 for automobile parts, and these data may include, for example, instructions for any application or method used to operate on the welding defect recognition device 800 for automobile parts, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or sent via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules can be a keyboard, a mouse or a button, etc. These buttons can be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the welding defect identification device 800 of the automotive parts and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 805 can include: a Wi-Fi module, a Bluetooth module or an NFC module.

[0054] In an exemplary embodiment, the welding defect recognition device 800 for automobile parts can be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), digital signal processors (Digital Signal Processing Device, referred to as DSP), digital signal processing devices (Digital Signal Processing Device, referred to as DSPD), programmable logic devices (Programmable Logic Device, referred to as PLD), field programmable gate arrays (Field Programmable Gate Array, referred to as FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned welding defect recognition method for automobile parts.

[0055] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned method for identifying welding defects of automobile parts are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 802 including program instructions, and the above-mentioned program instructions can be executed by the processor 801 of the welding defect identification device 800 for automobile parts to complete the above-mentioned method for identifying welding defects of automobile parts. Example 4

[0056] Corresponding to the above method embodiment, a readable storage medium is also provided in this embodiment. The readable storage medium described below and the welding defect identification method of automobile parts described above can be referred to each other.

[0057] A computer program is stored on the readable storage medium, and when the computer program is executed by the processor, the steps of the method for identifying welding defects of automobile parts in the above method embodiment are implemented.

[0058] The readable storage medium may specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or other readable storage medium that can store program codes.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0060] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for identifying welding defects of automobile parts, characterized in that: include: The three-dimensional structured light data of the area to be welded of the automobile parts and the acoustic wave signals generated during the welding process are obtained by using structured light vision sensors and high-frequency ultrasonic sensors, and the three-dimensional structured light data and the acoustic wave signals are preprocessed respectively, including filtering and denoising the three-dimensional structured light data to obtain edge and contour information for identifying the geometric features of the weld, and performing spectrum analysis on the acoustic wave signals to remove background noise and obtain characteristic frequency information for identifying internal defects of the weld; The thermal distribution image of the area to be welded of the automotive parts is captured by a thermal imaging camera, and the magnetic field change in the welding area is detected by magneto-optical imaging technology. The temperature of the thermal distribution image is calibrated to obtain the regional characteristic information for identifying thermal anomalies of welding defects. The magneto-optical imaging data is spatially filtered to obtain the magneto-optical signal characteristic information for identifying welding defects. Based on the edge and contour information, characteristic frequency information, regional characteristic information and magneto-optical signal characteristic information, a fused feature representation is obtained through deep learning algorithm processing. A deep learning model is trained to perform defect recognition and classification on the fused feature representation to obtain defect recognition results. The model can automatically learn the characteristics of welding defects and classify them into different categories, including cracks, pores and incomplete penetration. Based on the defect recognition results, the welding quality is evaluated, and the welding parameters are adjusted through the simulated annealing algorithm to generate a welding quality report, thereby identifying the welding defects of automotive parts.

2. The welding defect identification method of automobile parts according to claim 1 is characterized in that: The structured light vision sensor and the high-frequency ultrasonic sensor are used to obtain the three-dimensional structured light data of the area to be welded of the automobile parts and the acoustic wave signal generated during the welding process, and the three-dimensional structured light data and the acoustic wave signal are preprocessed respectively, including filtering and denoising the three-dimensional structured light data to obtain edge and contour information for identifying the geometric features of the weld, and performing spectrum analysis on the acoustic wave signal to remove background noise to obtain characteristic frequency information for identifying internal defects of the weld, including: The original three-dimensional structured light data of the area to be welded of the automobile parts collected by the structured light vision sensor is processed by Fourier transform to obtain the frequency domain representation of the weld; The frequency domain representation of the weld is processed using inverse Fourier transform and Gaussian filtering to obtain enhanced weld edge and contour information; The original sound wave signal generated during the welding process is collected by a high-frequency ultrasonic sensor and processed by short-time fractional Fourier transform to obtain a time-frequency domain representation, which includes analyzing the local spectrum characteristics of the sound wave signal by short-time fractional Fourier transform to provide time-frequency characteristics for identifying internal defects in welding; The time-frequency domain representation is processed by wavelet denoising to obtain the characteristic frequency information used to identify internal welding defects.

3. The welding defect identification method of automobile parts according to claim 1, characterized in that: The thermal distribution image of the area to be welded of the automobile parts is captured by a thermal imaging camera, and the magnetic field change of the welding area is detected by magneto-optical imaging technology. The temperature of the thermal distribution image is calibrated respectively to obtain regional feature information for identifying thermal anomalies of welding defects, and the magneto-optical imaging data is spatially filtered to obtain magneto-optical signal feature information for identifying welding defects, including: Based on the thermal distribution image of the area to be welded of the automotive parts captured by the thermal imaging camera, the regional characteristic information used to identify thermal anomalies of welding defects is obtained through calculations based on the blackbody radiation law and the Stefan-Boltzmann law. The calculation formula of the blackbody radiation law is as follows: In the formula, j ∗ is the total power (radiance) radiated per unit area of ​​a black body surface, T is the thermodynamic temperature of the black body itself, σ is the Stefan–Boltzmann constant, ϵ is the radiation coefficient of the black body; Based on the thermal distribution image, after 3D reconstruction processing using binocular vision technology, a 3D temperature distribution model for identifying welding defects is obtained using a pinhole camera model and a stereo matching algorithm. The stereo matching algorithm includes feature extraction, feature matching, and parallax calculation, and ultimately the three-dimensional coordinates of each pixel are obtained. Based on the 3D temperature distribution model, the thermal characteristics and defect locations of the welding area are obtained through the belief propagation algorithm, which optimizes the parameter estimation of the model through an iterative process; After the thermal characteristics and defect positions are processed by the spatial filter and the coherent optical image processing system, the enhanced magneto-optical signal characteristic information for identifying welding defects is obtained.

4. The welding defect identification method of automobile parts according to claim 1, characterized in that: The edge and contour information, characteristic frequency information, regional characteristic information and magneto-optical signal characteristic information are processed by a deep learning algorithm to obtain a fused feature representation, and a deep learning model is trained to perform defect recognition and classification on the fused feature representation to obtain a defect recognition result, which includes: According to the structured light feature vector, the ultrasonic feature vector, the thermal imaging feature vector and the magneto-optical feature vector, after being processed by the feature fusion layer, a fused feature representation is obtained; The fused feature representation is trained using a deep learning model, wherein the training process uses a cross entropy loss function and a back propagation algorithm to optimize the model parameters to obtain the trained model parameters; The trained model parameters are calculated using the forward propagation method, and the model outputs the predicted probability of each defect category. Finally, the defect type is determined based on the category with the highest probability, and the specific location of the defect is determined by combining image segmentation technology to obtain the defect recognition result.

5. The welding defect identification method of automobile parts according to claim 1, characterized in that: Based on the defect recognition results, the welding quality is evaluated, and the welding parameters are adjusted through the simulated annealing algorithm to generate a welding quality report, thereby identifying the welding defects of automotive parts, including: Based on the defect type, quantity and location in the defect identification results, combined with the welding process standards, the comprehensive score is calculated using defect density, defect distribution and defect severity indicators; The comprehensive score is processed by simulated annealing algorithm to obtain the optimized welding parameters, and the calculation formula is as follows: In the formula, Δ E is the energy difference between the new solution and the current solution, T is the current temperature, P (Δ E ) is the probability that the system accepts a new solution (even if its objective function value is worse than that of the current solution), and exp is a natural exponential function; The optimized welding parameters are processed through a feedback control mechanism, which includes monitoring the welding process and adjusting the welding equipment parameters in real time to obtain the final adjusted welding parameters, thereby identifying welding defects of automotive parts.

6. A welding defect recognition system for automobile parts, based on the welding defect recognition method for automobile parts according to claim 1, characterized in that: include: Processing module: used to obtain the three-dimensional structured light data of the area to be welded of the automobile parts and the acoustic wave signal generated during the welding process by using the structured light vision sensor and the high-frequency ultrasonic sensor, and pre-process the three-dimensional structured light data and the acoustic wave signal respectively, including filtering and denoising the three-dimensional structured light data to obtain the edge and contour information for identifying the geometric features of the weld, and performing spectrum analysis on the acoustic wave signal to remove the background noise and obtain the characteristic frequency information for identifying the internal defects of the weld; Calibration module: used to capture the thermal distribution image of the area to be welded of the automotive parts through a thermal imaging camera, and use magneto-optical imaging technology to detect the magnetic field change in the welding area, and perform temperature calibration on the thermal distribution image to obtain regional feature information for identifying thermal anomalies of welding defects, and perform spatial filtering on the magneto-optical imaging data to obtain magneto-optical signal feature information for identifying welding defects; Training module: used to obtain fused feature representations based on edge and contour information, characteristic frequency information, regional feature information and magneto-optical signal feature information after deep learning algorithm processing, and to obtain defect recognition results by training a deep learning model to perform defect recognition and classification on the fused feature representations. The model can automatically learn the characteristics of welding defects and classify them into different categories, including cracks, pores and incomplete penetration. Identification module: used to evaluate welding quality based on defect identification results, adjust welding parameters through simulated annealing algorithm, generate welding quality report, and thus identify welding defects of automotive parts.

7. The welding defect recognition system for automobile parts according to claim 6, characterized in that: The processing module includes: The first processing unit is used to obtain the frequency domain representation of the weld according to the original three-dimensional structured light data of the area to be welded of the automobile parts collected by the structured light vision sensor through Fourier transform processing; The second processing unit is used to process the frequency domain representation of the weld by using inverse Fourier transform and Gaussian filtering to obtain enhanced weld edge and contour information; The third processing unit is used to obtain the time-frequency domain representation of the original sound wave signal generated during the welding process based on the acquisition of the high-frequency ultrasonic sensor through short-time fractional Fourier transform, including analyzing the local spectrum characteristics of the sound wave signal through short-time fractional Fourier transform to provide time-frequency characteristics for identifying internal defects in welding; The fourth processing unit is used to process the time-frequency domain representation through wavelet denoising to obtain characteristic frequency information for identifying internal defects in welding.

8. The welding defect recognition system for automobile parts according to claim 6, characterized in that: The calibration module comprises: Calculation unit: used to obtain regional characteristic information for identifying thermal anomalies of welding defects based on the thermal distribution image of the area to be welded of the automobile parts captured by the thermal imaging camera, through the blackbody radiation law and the Stefan-Boltzmann law. The calculation formula of the blackbody radiation law is as follows: In the formula, j ∗ is the total power (radiance) radiated per unit area of ​​a black body surface, T is the thermodynamic temperature of the black body itself, σ is the Stefan–Boltzmann constant, ϵ is the radiation coefficient of the black body; The fifth processing unit is used to obtain a 3D temperature distribution model for identifying welding defects based on the thermal distribution image after 3D reconstruction processing using binocular vision technology, using a pinhole camera model and a stereo matching algorithm, wherein the stereo matching algorithm includes feature extraction, feature matching, and parallax calculation, and finally obtains the three-dimensional coordinates of each pixel point; Estimation unit: used to obtain the thermal characteristics and defect locations of the welding area based on the 3D temperature distribution model after being processed by the belief propagation algorithm, which optimizes the parameter estimation of the model through an iterative process; The sixth processing unit is used to process the thermal characteristics and defect positions through a spatial filter and a coherent optical image processing system to obtain enhanced magneto-optical signal characteristic information for identifying welding defects.

9. A welding defect identification device for automobile parts, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the welding defect identification method for automobile parts as claimed in any one of claims 1 to 5 when executing the computer program.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for identifying welding defects of automobile parts according to any one of claims 1 to 5 is implemented.

Images