A vibration-acoustics-based welding defect monitoring method and system

Abstract

The present application provides a method and system for welding defect monitoring based on vibration acoustics, which relates to the technical field of welding defect monitoring. The method includes: applying a low-frequency vibration signal to a monitoring target to obtain a response signal; performing spectral analysis on the response signal to obtain modal parameters; determining whether the modal parameters belong to a preset parameter range. If so, applying a real-time high-frequency ultrasonic signal to the monitoring target to obtain a real-time feedback signal, and extracting the signal characteristics of the real-time feedback signal; if not, outputting an alarm signal; determining whether the signal characteristics meet the expectations. If so, after a preset time interval, applying the low-frequency vibration signal to the monitoring target again; if not, outputting an alarm signal. The present application significantly improves the accuracy of the welding defect monitoring results by combining the low-frequency vibration signal and the high-frequency ultrasonic signal to perform double verification on the monitoring target.

Description

Technical Field

[0001] This application relates to the technical field of welding defect monitoring, and particularly to a welding defect monitoring method and system based on vibro-acoustics. Background Art

[0002] In industrial production, welding, as an important joining technology, is widely used in the manufacturing and assembly processes of various structures. However, various defects may occur during the welding process, such as cracks, incomplete penetration, etc. These defects will seriously affect the strength, toughness, and service life of the welded structure, and may even lead to structural failure and safety accidents.

[0003] Traditional welding defect monitoring methods mainly include visual monitoring, magnetic particle inspection, penetrant inspection, etc. Although these methods can detect welding defects to a certain extent, they have many limitations. For example, visual monitoring depends on the experience and skill level of the inspectors and is difficult to detect small or deeply hidden defects; magnetic particle inspection and penetrant inspection can only detect surface defects of welded parts after welding is completed, but cannot detect internal defects. Summary of the Invention

[0004] In order to improve the accuracy of welding defect monitoring, this application provides a welding defect monitoring method and system based on vibro-acoustics.

[0005] In a first aspect, this application provides a welding defect monitoring method based on vibro-acoustics, adopting the following technical solution:

[0006] A welding defect monitoring method based on vibro-acoustics includes the following steps:

[0007] Data acquisition: including first acquisition and second acquisition;

[0008] First acquisition: Apply a low-frequency vibration signal to the monitoring target to obtain a response signal;

[0009] Second acquisition: Apply a real-time high-frequency ultrasonic signal to the monitoring target to obtain a real-time feedback signal;

[0010] Data processing: including first processing and second processing;

[0011] First processing: Perform spectral analysis on the response signal to obtain modal parameters;

[0012] Second processing: Extract the signal characteristics of the real-time feedback signal;

[0013] Defect judgment: including parameter judgment and feature judgment;

[0014] Parameter judgment: Determine whether the modal parameter belongs to the preset parameter range. If so, perform the second acquisition step; if not, perform the output step;

[0015] Feature judgment: Determine whether the signal feature meets the expectation. If so, after an interval of the preset duration, perform the first acquisition step; if not, perform the output step;

[0016] Output: Output an alarm signal.

[0017] This application applies a low-frequency vibration signal to the monitoring target and obtains the corresponding response signal. The low-frequency vibration signal can excite the modal vibration of the monitoring target, and the obtained response signal contains information on the modal parameters of the monitoring target, such as modal frequency, modal damping, etc. Then, spectrum analysis is performed on the response signal to extract the frequency components and energy distribution in the response signal, and thus the modal parameters of the monitoring target are obtained. After that, by comparing the modal parameters with the preset parameter range, it can be preliminarily determined whether there are welding defects. If the modal parameters exceed the preset parameter range, it indicates that there are welding defects and an alarm signal needs to be issued. Otherwise, the second acquisition step needs to be performed to further check whether there are welding defects such as incomplete penetration in the monitoring target using high-frequency ultrasonic signals. The real-time high-frequency ultrasonic signal can obtain the internal detailed information of the welded structure. When the real-time high-frequency ultrasonic signal propagates in the monitoring target, phenomena such as reflection and scattering will occur when encountering welding defects, and these phenomena can be captured and used to analyze the position, size, and type of the welding defects. Then, a feedback signal is obtained, and the signal features of the feedback signal are extracted. These signal features include the intensity of the reflected wave, phase change, etc. By comparing these features with the expected values, it can be further determined whether there are welding defects in the monitoring target and the type and position of the welding defects. Then, it is determined whether the signal features meet the expectation. If the signal features do not meet the expectation, it indicates that there are welding defects in the monitoring target, and then an alarm signal is issued. This application uses a dual judgment mechanism to monitor welding defects, improving the accuracy and reliability of the welding defect monitoring results.

[0018] Optionally, before performing the data acquisition step, the method further includes:

[0019] Third acquisition: Acquire historical images during welding and add category labels to the historical images;

[0020] Fourth acquisition: Acquire real-time images of the monitoring target;

[0021] First modeling and training: Establish a CNN model and train the CNN model using the historical images and the added category labels to obtain the trained CNN model;

[0022] First prediction: Input the real-time image into the trained CNN model to obtain a prediction result;

[0023] Result judgment: Determine whether the predicted result meets the expectation. If so, execute the steps of the first acquisition; if not, execute the output steps.

[0024] This application provides a data basis for subsequent model training by collecting historical images during welding and adding category labels (such as normal welding, cracks, incomplete penetration, etc.) to these historical images. Then, a CNN model is established and trained using the historical images and category labels, enabling the CNN model to learn the deep features in the images. Through training, the CNN model can accurately identify the types of defects during the welding process, improving the accuracy of monitoring. Subsequently, the real-time image is input into the trained CNN model, and the CNN model can predict the image based on the learned features to determine whether there are welding defects in the welding process, reducing the subjectivity and error of human judgment. Before performing the data acquisition steps, the category label of the real-time image is predicted through the CNN model to determine whether the predicted result meets the expectation. If the predicted result meets the expectation (i.e., there are no welding defects), continue to execute the first acquisition step to continue monitoring for the presence of welding defects. If the predicted result does not meet the expectation (i.e., there are defects in the welding process), directly execute the output step to send an alarm signal. This application realizes multi-dimensional monitoring of welding defects by combining visual monitoring with vibration acoustics and ultrasonic detection technologies. Visual monitoring can intuitively reflect the surface conditions of the welding process, while vibration acoustics and ultrasonic monitoring can deeply detect internal defects in the welding structure. This comprehensive monitoring method improves the comprehensiveness and accuracy of monitoring, providing a more reliable guarantee for welding quality.

[0025] Optionally, after performing the steps of the fourth acquisition and before performing the steps of the first modeling and training, it further includes:

[0026] Third processing: Perform grayscale processing on the real-time image to obtain the grayscale value of each pixel point in the real-time image;

[0027] Fourth processing: Use the region growing algorithm to segment the grayscale real-time image to obtain a target image block containing the monitoring target and a remaining image block not containing the monitoring target;

[0028] Reset grayscale value: Set the grayscale value of the pixel points corresponding to the remaining image block in the real-time image to 0 to obtain a new real-time image.

[0029] This application first performs grayscale processing on the real-time image, aiming to remove the interference of color information on image processing and retain the morphological and structural features of the image. Then, the region growing algorithm is used to segment the grayscale real-time image, and the image is divided into different regions according to the similarity of pixel points, namely the target image block containing the monitoring target and the remaining image block not containing the monitoring target. Then, in the real-time image, the grayscale values of the pixel points corresponding to the remaining image block are set to 0 to remove the background information of the real-time image. By removing the image area that does not contain the monitoring target, the interference of background noise on target recognition can be reduced, and the stability and accuracy of target recognition can be improved.

[0030] Optionally, after the step of performing the third processing and before the step of performing the fourth processing, it further includes:

[0031] Fifth processing: Integrate the grayscale values greater than the preset grayscale threshold into the first data set, and integrate the remaining grayscale values into the second data set;

[0032] First calculation: Calculate the degree of difference between the first data set and the second data set, and the calculation model of the degree of difference is as follows:

[0033] ;

[0034] where r represents the degree of difference; A is the number of grayscale values in the first data set; B is the number of grayscale values in the second data set; is the average value of all grayscale values in the first data set; is the average value of all grayscale values in the second data set;

[0035] Difference judgment: Judge whether the degree of difference is less than the preset difference threshold. If so, perform the step of grayscale value judgment; if not, perform the step of fourth processing;

[0036] Grayscale value judgment: Judge whether A is greater than B. If so, perform the step of first adjustment; if not, perform the step of second adjustment;

[0037] Adjustment threshold: Includes first adjustment and second adjustment;

[0038] First adjustment: Calculate the difference between each grayscale value in the first data set and the preset grayscale threshold, denoted as the first difference, and select the grayscale value corresponding to the minimum first difference as the new preset grayscale threshold, and perform the step of fifth processing;

[0039] Second adjustment: Calculate the difference between the grayscale values in the second data set and the preset grayscale threshold, denoted as the second difference, and select the grayscale value corresponding to the minimum second difference as the new preset grayscale threshold, and perform the step of fifth processing.

[0040] In this application, the gray values are first divided into two groups by a preset gray threshold, and based on this, a first data set and a second data set are respectively constructed. Then, a calculation model of the degree of difference is used to quantify the degree of difference between the two data sets, so as to quantify the difference between the two groups of gray values. After that, the degree of difference is compared with a preset difference threshold. When the degree of difference is less than the preset difference threshold, it indicates that the difference between the highlighted area and the dark area in the real-time image is not obvious enough, and the preset difference threshold needs to be further adjusted. Therefore, this application selects a new preset gray threshold according to the difference between the gray values in the first data set and the second data set and the preset gray threshold, so that the degree of difference can be greater than the preset difference threshold. This application can make the new gray threshold more in line with the actual situation of the image data, improving the accuracy and efficiency of subsequent processing.

[0041] Optionally, after the step of performing the difference judgment and before the step of performing the fourth processing, it further includes:

[0042] Fifth acquisition: Record the degree of difference calculated in the step of the first calculation as the first data, and integrate all the first data into a third data set;

[0043] Extract the maximum and minimum values: Extract the largest first data in the third data set, record it as the second data, and obtain the preset gray threshold corresponding to the second data, record it as the third data;

[0044] First search: Search in the first data set and the second data set for the gray value with the smallest difference from the third data, and record it as the fourth data;

[0045] Determine the seed point: Use the pixel point corresponding to the fourth data as the seed point in the step of the fourth processing.

[0046] This application integrates the degree of difference obtained after each execution of the first calculation step into a third data set. Then, it finds the largest degree of difference (i.e., the second data) in the third data set and obtains the corresponding preset gray threshold (i.e., the third data) to obtain the preset gray threshold corresponding to the largest degree of difference. The largest degree of difference reflects the most significant or important feature changes in the image data. Then, it searches in the first data set and the second data set for the gray value closest to the third data (i.e., the fourth data), and then uses the pixel point corresponding to the fourth data as the seed point in the step of the fourth processing, thereby improving the accuracy and efficiency of image segmentation. By deeply analyzing the degree of difference data, this application extracts key feature points and determines appropriate seed points, which not only improves the accuracy and efficiency of image processing, but also helps to discover potential rules and features in the image data, providing more critical information for subsequent image segmentation.

[0047] Optionally, after the step of performing the first search and before the step of determining the seed point, it further includes:

[0048] Quantity judgment: Judge whether the quantity of the fourth data is equal to one. If so, execute the step of determining the seed point; if not, execute the integration step;

[0049] Integration: Integrate all the fourth data into a fourth data set;

[0050] First acquisition: Acquire the pixel point corresponding to the k-th fourth data, denoted as the first pixel point, and acquire the eight-neighborhood pixel points of the first pixel point, denoted as the second pixel point;

[0051] Second acquisition: Acquire the gray value of the second pixel point, denoted as the fifth data;

[0052] Noise point judgment: Use the k-th fourth data and the fifth data to judge whether the first pixel point is a noise point. If so, execute the first deletion step; if not, execute the iteration step;

[0053] First deletion: Delete the k-th fourth data from the fourth data set;

[0054] Iteration: Take the (k + 1)-th fourth data as the new k-th fourth data, and execute the quantity judgment step.

[0055] This application determines the execution order of subsequent steps according to the quantity of the fourth data. When the quantity of the fourth data is equal to one, it means that there is only one most likely seed point candidate, and directly execute the step of determining the seed point. When the quantity of the fourth data is greater than one, it means that there may be noise points. Then, all the fourth data are integrated into a fourth data set, and the pixel points are screened to delete the noise points. After that, this application acquires the pixel point corresponding to the k-th fourth data (i.e., the first pixel point) and its eight-neighborhood pixel points (i.e., the second pixel point). Then, the k-th fourth data is compared with the fifth data to judge whether the first pixel point is a noise point. If the first pixel point is a noise point, delete the k-th fourth data from the fourth data set to remove the possible noise points and improve the accuracy of subsequent image segmentation. Then, take the (k + 1)-th fourth data as the new k-th fourth data and re-execute the quantity judgment step to analyze and process all the fourth data one by one until all possible noise points are removed. This application effectively removes the possible noise points by carefully analyzing the fourth data and the fifth data, making the finally determined seed point more accurate and reliable. This application not only improves the accuracy and efficiency of image segmentation, but also helps to reduce the interference of noise on the real-time image segmentation result.

[0056] Optionally, after executing the iteration step and before executing the step of determining the seed point, it further includes:

[0057] Third acquisition: Acquire the remaining fourth data in the fourth data set, denoted as the sixth data;

[0058] Fourth acquisition: Acquire the sixth data that can form a closed figure, denoted as the seventh data;

[0059] Second deletion: Retain the seventh data in the fourth dataset and delete the remaining fourth data;

[0060] In the step of determining the seed point, use the pixel point corresponding to any one of the seventh data in the second deletion step as the seed point.

[0061] This application acquires the remaining fourth data in the fourth dataset and denotes it as the sixth data. Then, it acquires the sixth data that can form a closed figure and denotes it as the seventh data. After that, it retains the fourth data corresponding to the seventh data in the fourth dataset and deletes the remaining fourth data. Then, it uses the pixel point corresponding to any one of the seventh data (i.e., the fourth data screened by the second deletion step) in the fourth dataset as the seed point, so as to select a more suitable starting point for subsequent processing, which helps to improve the efficiency and accuracy of image processing. By adopting the above scheme, this application can determine whether there is connectivity between a pixel point and its surrounding pixel points in terms of the position distribution of the pixel points, so as to determine whether it is suitable as a seed point.

[0062] Optionally, after the step of performing feature judgment and before the step of output, it further includes:

[0063] Sixth collection: Collect the historical feedback signal of the monitoring target;

[0064] Add label: Add a type label to the historical feedback signal, and denote the historical feedback signal after adding the type label as the first signal;

[0065] Second modeling and training: Establish an RNN model, and use the first signal to train the RNN model to obtain a trained RNN model;

[0066] Recognition: Input the real-time feedback signal into the trained RNN model to obtain the predicted type label;

[0067] Label judgment: Judge whether the predicted type label is an abnormal label. If so, perform the output step; if not, perform the first collection step.

[0068] This application collects historical feedback signals of the monitoring target, then adds type tags (such as normal, abnormal, etc.) to the historical feedback signals, then establishes an RNN (Recurrent Neural Network) model, and uses the first signal (i.e., the historical feedback signal after adding type tags) to train the RNN model so that the trained RNN model can classify real-time feedback signals. Then, by inputting the real-time feedback signal into the trained RNN model, the type prediction result of the signal can be obtained, that is, whether it is an abnormal signal. If the prediction result is abnormal, the output step is executed; if the prediction result is normal, the first collection step is returned to continue monitoring or other relevant operations are executed. This application can learn and identify abnormal behaviors or states of the monitoring target based on historical data, so as to take corresponding measures in a timely manner.

[0069] Optionally, after performing the second modeling and training step and before performing the recognition step, it further includes:

[0070] Seventh collection: Collect the historical high-frequency ultrasonic signals corresponding to the historical feedback signals, and the difference between the time of collecting the historical feedback signals and the time of collecting the historical high-frequency ultrasonic signals;

[0071] Model expansion: In the trained RNN model, add multiple hidden layers to obtain an expanded RNN model. Denote the added hidden layers in the expanded RNN model as the first hidden layer, and the original hidden layers in the expanded RNN model as the second hidden layer;

[0072] Third training: Fix the second hidden layer and use the historical high-frequency ultrasonic signals and the difference to train the first hidden layer to obtain the RNN model after retraining;

[0073] Model update: Use the RNN model after retraining as the new trained RNN model, and perform the recognition step;

[0074] In the recognition step, the difference between the time of collecting the real-time feedback signal and the time of collecting the real-time high-frequency ultrasonic signal is also input into the new trained RNN model.

[0075] This application collects historical high-frequency ultrasonic signals corresponding to historical feedback signals, as well as the time differences between the collection times of these signals. If there are welding defects, the time difference between the emission of high-frequency ultrasonic signals and the reception of feedback signals is different. Therefore, this application also retrains the trained RNN model by collecting the time difference between the two, enabling the retrained RNN model to further determine whether there are welding defects based on the time difference between the feedback signal and the high-frequency ultrasonic signal. This application also adds multiple hidden layers to the trained RNN model and fixes the second hidden layer (i.e., the original hidden layer) in the RNN model. Only the first hidden layer (i.e., the newly added hidden layer) is trained using the historical high-frequency ultrasonic signal and the time difference, thereby introducing new signal types and features while maintaining the original RNN model structure and learning results. By fixing the second hidden layer, the useful information already learned by the original RNN model can be not destroyed during the retraining process, and the focus is on training the first hidden layer of the RNN model to enable it to focus on learning new signal features and patterns. This application improves the accuracy and reliability of welding defect monitoring results by introducing high-frequency ultrasonic signals and time differences as new input features and correspondingly expanding and training the RNN model, enabling the RNN model to learn richer information.

[0076] In a second aspect, this application provides a welding defect monitoring system based on vibro-acoustics, adopting the following technical solutions:

[0077] A welding defect monitoring system based on vibro-acoustics includes:

[0078] A data acquisition module, including a first acquisition unit and a second acquisition unit;

[0079] The first acquisition unit is used to apply a low-frequency vibration signal to the monitoring target and obtain a response signal;

[0080] The second acquisition unit is used to apply a real-time high-frequency ultrasonic signal to the monitoring target and obtain a real-time feedback signal;

[0081] A data processing module, including a first processing unit and a second processing unit;

[0082] The first processing unit is used to perform spectral analysis on the response signal to obtain modal parameters;

[0083] The second processing unit is used to extract the signal features of the real-time feedback signal;

[0084] A defect judgment module, including a parameter judgment unit and a feature judgment unit;

[0085] The parameter judgment unit is used to judge whether the modal parameters belong to a preset parameter range. If they belong to the preset parameter range, then the second acquisition unit is triggered; otherwise, the output module is triggered;

[0086] A feature judgment unit, configured to judge whether the signal feature meets the expectation. If it meets the expectation, then after a preset time interval, trigger the first acquisition unit; otherwise, trigger the output module.

[0087] An output module, configured to output an alarm signal.

[0088] In this application, a low-frequency vibration signal is applied by the first acquisition unit to obtain a response signal for preliminary evaluation, a real-time high-frequency ultrasonic signal is applied by the second acquisition unit to obtain detailed feedback, the data processing module performs spectral analysis and feature extraction on the signal, the defect judgment module judges the state of the monitoring target according to the preset parameter range and signal features, and triggers the output module to output an alarm signal when an abnormality is found. This application combines multiple signals for continuous monitoring of welding defects, improving the accuracy of welding defect monitoring.

[0089] In summary, this application includes at least one of the following beneficial technical effects:

[0090] 1. This application uses a dual judgment mechanism to monitor welding defects, improving the accuracy and reliability of welding defect monitoring.

[0091] 2. By combining visual monitoring with vibration acoustics and ultrasonic monitoring technologies, this application realizes multi-dimensional monitoring of welding defects. Visual monitoring can intuitively reflect the surface condition of the welding process, while vibration acoustics and ultrasonic monitoring technologies can deeply detect internal defects in the welding structure. This comprehensive monitoring method improves the comprehensiveness and accuracy of monitoring, providing a more reliable guarantee for welding quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 is a flowchart of Embodiment 1 of this application;

[0093] Figure 2 is a flowchart of S01 Third Acquisition to S08 Result Judgment of Embodiment 2 of this application;

[0094] Figure 3 is a flowchart of S51 Fifth Processing to S66 Adjust Threshold of Embodiment 2 of this application;

[0095] Figure 4 is a flowchart of S71 Third Acquisition to S73 Second Deletion of Embodiment 2 of this application;

[0096] Figure 5 is a flowchart of Embodiment 3 of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0097] The following Figures 1 to 5 further elaborates on this application in detail.

[0098] Embodiment 1: This embodiment discloses a method for monitoring welding defects based on vibro-acoustics. Refer to Figure 1 , the monitoring method includes: S1 data acquisition, S2 data processing, S3 defect judgment, and S4 output. S1 data acquisition includes two stages: the first acquisition (applying a low-frequency vibration signal to obtain a response signal) and the second acquisition (applying a real-time high-frequency ultrasonic signal to obtain a feedback signal). Subsequently, S2 data processing (including spectral analysis and feature extraction) is performed, and then S3 defect judgment (judging whether the modal parameters and signal features meet the preset standards) is used to decide whether to continue monitoring or output an alarm signal. The process of this embodiment is as follows:

[0099] S1 data acquisition includes S11 the first acquisition and S12 the second acquisition.

[0100] S11 the first acquisition: Apply a low-frequency vibration signal to the monitoring target and obtain a response signal.

[0101] The low-frequency vibration signal is used to excite the modal response of the monitoring target. The mode is the vibration mode of the welded structure at a specific frequency and is closely related to the physical properties of the welded structure (such as mass, stiffness, and damping). By applying a low-frequency vibration signal and measuring the resulting response signal, the modal behavior of the monitoring target can be captured.

[0102] The response signal contains information about the modal parameters of the monitoring target, such as modal frequency, modal damping, and modal shape.

[0103] S12 the second acquisition: Apply a real-time high-frequency ultrasonic signal to the monitoring target and obtain a real-time feedback signal.

[0104] The real-time high-frequency ultrasonic signal is used to perform a more detailed detection of the monitoring target. The high-frequency ultrasonic signal has the characteristics of strong penetration and high resolution, and can penetrate deep into the material and reflect its microscopic structure and defect conditions.

[0105] By applying a real-time high-frequency ultrasonic signal to the monitoring target and receiving the resulting real-time feedback signal, detailed information about the internal state of the monitoring target can be captured. These detailed information include the density, thickness, cracks, slag inclusions, etc. of the welding material, so as to judge whether there are welding defects in the monitoring target.

[0106] S2 data processing includes S21 the first processing and S22 the second processing.

[0107] S21 the first processing: Perform spectral analysis on the response signal to obtain modal parameters.

[0108] The spectral analysis technology converts the response signal from the time domain to the frequency domain, thereby revealing the intensity of different frequency components in the signal. Through spectral analysis, modal parameters in the response signal, such as modal frequency and modal damping, can be extracted.

[0109] S22 Second processing, extracting signal features of the real-time feedback signal, where the signal features include: amplitude, frequency, phase, etc.

[0110] S3 Defect judgment, including S31 parameter judgment and S32 feature judgment.

[0111] S31 Parameter judgment, respectively judging whether various modal parameters belong to the preset parameter range. If so, it indicates that the monitoring target may be in a normal state, and then S12 Second acquisition is executed to further determine whether there are welding defects in the monitoring target; if not, it indicates that there are defects or abnormalities in the monitoring target, and S4 Output is executed.

[0112] S32 Feature judgment, judging whether the signal features meet the expectations. If so, it indicates that the monitoring target is in a normal state, and after an interval of the preset duration, S11 First acquisition is executed; if not, it indicates that there are defects or abnormalities in the monitoring target, and S4 Output is executed.

[0113] The standard for the signal features to meet the expectations is that the signal features such as amplitude, frequency, and phase are within the preset range.

[0114] S4 Output, outputting an alarm signal.

[0115] This embodiment includes S1 Data acquisition (including S11 First acquisition of obtaining a response signal by applying a low-frequency vibration signal to the monitoring target, and S12 Second acquisition of obtaining a real-time feedback signal by applying a real-time high-frequency ultrasonic signal), S2 Data processing (including S21 First processing of performing spectral analysis on the response signal to obtain modal parameters, and S22 Second processing of extracting signal features of the real-time feedback signal), S3 Defect judgment (including S31 Parameter judgment based on whether the modal parameters are within the preset parameter range and S32 Feature judgment on whether the signal features meet the expectations), and S4 Output to accurately determine whether there are welding defects in the monitoring target.

[0116] Embodiment 2: The difference between this embodiment and Embodiment 1 is that, referring to Figure 2 , before performing S1 data acquisition, the method further includes:

[0117] S01 Third acquisition, acquiring historical images during different welding processes and adding category labels to the historical images. These historical images include images of different welding materials under different welding parameters, and the category labels added to these historical images include normal welding labels and abnormal welding labels. The abnormal welding labels also include: cracks, pores, slag inclusions, etc.

[0118] S02 Fourth acquisition, acquiring real-time images of the monitoring target.

[0119] S03 Third process: perform grayscale processing on the real-time image to obtain the grayscale value of each pixel point in the real-time image.

[0120] S04 Fourth process: use the region growing algorithm to segment the grayscale real-time image to obtain multiple image blocks, including: target image blocks containing the monitoring target and remaining image blocks not containing the monitoring target.

[0121] S05 Reset grayscale value: set the grayscale value of the pixel points corresponding to the remaining image blocks in the real-time image to 0, so as to more clearly highlight the target image blocks containing the monitoring target, and at the same time reduce the interference of background information. Take the image with the grayscale value set as the new real-time image.

[0122] S06 First modeling and training: establish a CNN model, and use historical images and added class labels to train the CNN model to obtain the trained CNN model. Through training, a CNN model that can accurately identify the class of the monitoring target (such as whether the welding state is normal, the type of welding defect, etc.) can be obtained.

[0123] S07 First prediction: input the new real-time image in S05 reset grayscale value into the trained CNN model to obtain the prediction result.

[0124] S08 Result judgment: judge whether the prediction result meets the expectation. If so, execute S11 First collection to continue monitoring whether there are defects in the welded structure; if not, execute S4 output.

[0125] In this embodiment, first, welding historical images are collected and labeled with categories, then real-time images of the monitoring target are obtained, then the real-time images are subjected to grayscale processing, then the region growing algorithm is used to segment the images to distinguish the target from the background, the grayscale value of the background is set to 0 to highlight the target area, then a CNN model is established and trained to identify the state of the monitoring target, the real-time image is input into the trained model to obtain the prediction result, and then it is decided whether to continue collecting or output an alarm signal according to whether the prediction result meets the expectation. Based on applying low-frequency vibration signals and high-frequency ultrasonic signals to the monitoring target, this embodiment further improves the monitoring ability of welding defects by combining historical graphics and real-time images, so as to achieve the function of accurately monitoring welding defects.

[0126] Refer to Figure 3 , in other embodiments, after performing S03 third process and before performing S04 fourth process, it further includes:

[0127] S51 Fifth process: based on the grayscale value of each pixel point in the real-time image in S03 third process, integrate the grayscale values greater than the preset grayscale threshold into the first data set, and integrate the remaining grayscale values into the second data set.

[0128] S52 First calculation, calculate the difference degree between the first data set and the second data set. The calculation model of the difference degree is as follows:

[0129] ;

[0130] where r represents the difference degree; A is the number of gray values in the first data set; B is the number of gray values in the second data set; is the average value of all gray values in the first data set; is the average value of all gray values in the second data set.

[0131] S53 Difference judgment, judge whether the difference degree calculated in S52 First calculation is less than the preset difference threshold. If so, it indicates that the preset gray threshold currently set is not sufficient to distinguish the target area and the background of the real-time image, and S65 Gray value judgment needs to be executed to adjust the preset gray threshold; if not, execute S54 Fifth acquisition.

[0132] S54 Fifth acquisition, record the difference degree calculated in S52 First calculation as the first data, and obtain the first data calculated after each execution of S52 First calculation. Integrate all the first data into the third data set.

[0133] S55 Extract the maximum value, extract the largest first data in the third data set, record it as the second data, and obtain the preset gray threshold corresponding to the second data, record it as the third data.

[0134] S56 First search, calculate the difference between the gray value in the first data set and the third data respectively, calculate the difference between the gray value in the second data set and the third data respectively, obtain the minimum value among all the differences, and determine the gray value corresponding to the minimum value. Record the gray value corresponding to the minimum value as the fourth data.

[0135] S57 Quantity judgment, judge whether the quantity of the fourth data is equal to one. If so, execute S64 Determine the seed point; if not, execute S58 Integration.

[0136] S58 Integration, integrate all the fourth data into the fourth data set.

[0137] S59 First acquisition, acquire the pixel point corresponding to the k-th fourth data, record it as the first pixel point, and determine the eight-neighborhood pixel points of the first pixel point, record it as the second pixel point.

[0138] S60 Second acquisition, acquire the gray value of the second pixel point, record it as the fifth data.

[0139] S61 Noise point judgment: Use the k-th fourth data and the fifth data to judge whether the first pixel point is a noise point. If so, execute S62 First deletion; if not, execute S63 Iteration.

[0140] The method for using the k-th fourth data and the fifth data to judge whether the first pixel point is a noise point is as follows: Judge whether the brightness of the first pixel point is significantly different from the brightness of the second pixel point. If so, consider the first pixel point as a noise point; otherwise, consider the first pixel point not as a noise point.

[0141] S62 First deletion: Delete the k-th fourth data in the fourth dataset to remove the noise point.

[0142] S63 Iteration: Take the (k + 1)-th fourth data as the new k-th fourth data, and execute S57 Quantity judgment until the gray values of all pixel points have been compared with the gray values of the eight-neighbor pixel points.

[0143] S64 Determine the seed points: Take the pixel points corresponding to the fourth data that have not been deleted in the fourth dataset as the seed points in S04 Fourth processing.

[0144] S65 Gray value judgment: Judge whether A is greater than B. If so, it indicates that the sample quantity in the first dataset is greater than the sample data in the second dataset, and then execute S661 First adjustment; if not, execute S662 Second adjustment.

[0145] S66 Adjust the threshold, including S661 First adjustment and S662 Second adjustment.

[0146] S661 First adjustment: Calculate the difference between each gray value in the first dataset and the preset gray threshold, denoted as the first difference. Select the gray value corresponding to the minimum first difference as the new preset gray threshold, and execute S51 Fifth processing.

[0147] S662 Second adjustment: Calculate the difference between the gray value in the second dataset and the preset gray threshold, denoted as the second difference. Select the gray value corresponding to the minimum second difference as the new preset gray threshold, and execute S51 Fifth processing.

[0148] This embodiment performs a fifth process based on the grayscale values of pixel points in a real-time image, dividing the grayscale values into a first data set and a second data set; then calculates the degree of difference between the two data sets, and determines whether to adjust a preset grayscale threshold according to the comparison result between the degree of difference and the preset threshold; through a series of calculations, judgments, and iterative processes, identifies and removes noise points, and determines seed points; finally, according to the sample numbers of the first data set and the second data set, selects different adjustment strategies to optimize the preset grayscale threshold, including respectively calculating the differences between the grayscale values in the two data sets and the preset grayscale threshold, and selecting the grayscale value with the smallest difference as the new preset grayscale threshold, and then re-executing the steps of the fifth process to improve the accuracy and efficiency of image processing.

[0149] Referring to Figure 4 , in other embodiments, after performing the iteration of S63 and before performing S64 to determine the seed points, it further includes:

[0150] S71 Third acquisition, acquires the remaining fourth data in the fourth data set, denoted as the sixth data.

[0151] S72 Fourth acquisition, marks the pixel points corresponding to the sixth data in the real-time image, acquires the pixel points that can form a closed figure, and denotes the corresponding sixth data as the seventh data.

[0152] S73 Second deletion, retains the seventh data in the fourth data set, and deletes the remaining fourth data.

[0153] In S64 to determine the seed points, any pixel point corresponding to one of the seventh data in the fourth data set in S73 Second deletion is used as the seed point.

[0154] This embodiment first acquires the remaining data in the fourth data set as the sixth data, then screens out the partial data corresponding to the pixel points that can form a closed figure as the seventh data, and then only retains the seventh data in the fourth data set and deletes the remaining data. Finally, in the step of determining the seed points, any one of the pixel points corresponding to the retained seventh data is selected as the seed point for subsequent image processing or data analysis tasks. This embodiment further screens the fourth data, further improving the efficiency of image processing.

[0155] Embodiment 3: Referring to Figure 5 , the difference between this embodiment and Embodiment 2 is that after performing S32 feature judgment and before performing S4 output, it further includes:

[0156] S81 Sixth acquisition, acquires the historical feedback signal of the monitoring target.

[0157] S82 Add a label, add a type label to the historical feedback signal, and denote the historical feedback signal after adding the type label as the first signal.

[0158] S83 Second modeling and training, establish an RNN model, and use the first signal to train the RNN model to obtain the trained RNN model.

[0159] S84 Seventh acquisition, acquire the historical high-frequency ultrasonic signals corresponding to the historical feedback signal, record their timestamps, and calculate the difference between the two timestamps.

[0160] S85 Model expansion, in the trained RNN model, add multiple hidden layers to obtain the expanded RNN model. Denote the added hidden layers in the expanded RNN model as the first hidden layer, and denote the original hidden layers in the expanded RNN model as the second hidden layer.

[0161] S86 Third training, fix the second hidden layer, and use the historical high-frequency ultrasonic signal and the difference to train the first hidden layer to obtain the retrained RNN model.

[0162] S87 Model update, use the retrained RNN model as the new trained RNN model.

[0163] S88 Recognition, input the real-time feedback signal, the time of collecting the real-time feedback signal, and the difference between the time of collecting the real-time high-frequency ultrasonic signal into the trained RNN model to obtain the predicted type label.

[0164] S89 Label judgment, judge whether the predicted type label is an abnormal label. If so, execute S4 output; if not, execute S11 First acquisition.

[0165] In this embodiment, first, the historical feedback signal of the monitoring target is acquired and a type label is added to it. Then, an RNN model is established and trained using these labeled signals. After that, the historical high-frequency ultrasonic signal associated with the feedback signal and the time difference are acquired, the RNN model is expanded, and some hidden layers are fixed, and only the newly added hidden layers are trained for the high-frequency ultrasonic signal and the time difference. After updating the model, the real-time feedback signal and the time difference are input into the model for type recognition, and it is judged whether to execute the abnormal handling or continue the monitoring operation according to the prediction result. In this embodiment, by judging the label of the real-time feedback signal, it is determined whether the feedback signal is an abnormal signal. If so, an alarm signal is output; otherwise, the steps of the first acquisition are executed.

[0166] Embodiment 4: This embodiment discloses a welding defect monitoring system based on vibroacoustics. The monitoring system includes:

[0167] A data acquisition module, including a first acquisition unit and a second acquisition unit.

[0168] A first acquisition unit is configured to apply a known low-frequency vibration signal to a monitoring target and then acquire the response of the target to this signal (i.e., acquire the corresponding signal).

[0169] A second acquisition unit is configured to apply a real-time high-frequency ultrasonic signal to the monitoring target and acquire a real-time feedback signal.

[0170] A data processing module includes a first processing unit and a second processing unit.

[0171] The first processing unit is communicatively connected to the first acquisition unit and is configured to perform spectral analysis on the response signal to obtain modal parameters (such as modal frequency, modal damping, etc.).

[0172] The second processing unit is communicatively connected to the second acquisition unit and is configured to extract the signal features of the real-time feedback signal.

[0173] A defect judgment module includes a parameter judgment unit and a feature judgment unit.

[0174] The parameter judgment unit is communicatively connected to the first processing unit and the second acquisition unit and is configured to judge whether the modal parameters belong to a preset parameter range. If they belong to the preset parameter range, then the second acquisition unit is triggered; otherwise, the output module is triggered. By comparing the deviation degree between the modal parameters and the preset range, the parameter judgment unit can evaluate whether there are abnormalities or defects in the welding structure of the monitoring target. If the modal parameters are within the preset range, it is considered that the monitoring target is in a normal state; otherwise, it is considered that there are potential problems.

[0175] The feature judgment unit is communicatively connected to the second processing unit and the first acquisition unit and is configured to judge whether the signal features meet the expectations. If they meet the expectations, then after a preset time interval, the first acquisition unit is triggered; otherwise, the output module is triggered. By comparing and analyzing the signal features with the expected values, the feature judgment unit can detect whether there are abnormal changes or potential defects in the monitoring target. If the signal features match the expected values or the deviation is within an acceptable range, it is considered that the monitoring target is in a normal state; otherwise, it is considered that there are abnormalities.

[0176] An output module is communicatively connected to the defect judgment module and is configured to output an alarm signal.

[0177] This embodiment integrates a data acquisition module (including a first acquisition unit for applying a low-frequency vibration signal to obtain a response signal and a second acquisition unit for applying a real-time high-frequency ultrasonic signal to obtain a feedback signal), a data processing module (including a first processing unit for performing spectral analysis to obtain modal parameters and a second processing unit for extracting real-time signal features), a defect judgment module (including a parameter judgment unit for judging whether the modal parameters are abnormal to trigger corresponding operations and a feature judgment unit for evaluating whether the signal features meet the expectations to determine subsequent steps), and an output module, so as to achieve accurate monitoring of welding defects.

[0178] The above are all preferred embodiments of this application. The protection scope of this application is not limited thereby. Therefore, all equivalent changes made according to the structure, shape, and principle of this application shall be covered within the protection scope of this application.

Claims

1. A welding defect monitoring method based on vibroacoustics, characterized in that: include: Collect historical images of welding and add category labels to the historical images; Collect real-time images of monitoring targets; Perform grayscale processing on the real-time image to obtain the grayscale value of each pixel in the real-time image; Calculating the degree of difference: integrating the grayscale values ​​greater than a preset grayscale threshold into a first data set, and integrating the remaining grayscale values ​​into a second data set; calculating the degree of difference between the first data set and the second data set; Difference judgment: judge whether the difference is less than the preset difference threshold, if so, execute the gray value judgment step; If not, the calculated difference degree is recorded as the first data, and all the first data are integrated into the third data set; Extracting the largest first data from the third data set, recording it as the second data, and obtaining a preset grayscale threshold corresponding to the second data, recording it as the third data; Find the gray value with the smallest difference with the third data in the first data set and the second data set, and record it as the fourth data; Quantity judgment: judging whether the quantity of the fourth data is equal to one; If yes, then the step of determining the seed point is performed; If not, integrate all the fourth data into a fourth data set; obtain the pixel point corresponding to the kth fourth data, record it as the first pixel point, obtain the eight neighboring pixel points of the first pixel point, record it as the second pixel point; obtain the gray value of the second pixel point, record it as the fifth data; Noise point judgment: using the kth fourth data and the fifth data to judge whether the first pixel is a noise point; If yes, then delete the kth fourth data in the fourth data set; If not, the k+1th fourth data is used as the new kth fourth data, and the quantity determination step is performed until all the pixels are compared with the grayscale values ​​of the eight neighboring pixels; Determine a seed point: use a pixel point corresponding to the fourth data as a seed point in the step of updating the image; Gray value judgment: judging whether the number of gray values ​​in the first data set is greater than the number of gray values ​​in the second data set; If yes, then respectively calculate the difference between each gray value in the first data set and the preset gray threshold, record it as the first difference, select the gray value corresponding to the minimum first difference as the new preset gray threshold, and execute the step of calculating the degree of difference; If not, then the difference between the grayscale value in the second data set and the preset grayscale threshold is calculated, recorded as the second difference, the grayscale value corresponding to the minimum second difference is selected as the new preset grayscale threshold, and the step of calculating the degree of difference is performed; Update image: Use region growing algorithm to segment the grayscale real-time image to obtain target image blocks containing the monitored target and remaining image blocks that do not contain the monitored target; set the grayscale values ​​of the corresponding pixels of the remaining image blocks in the real-time image to 0 to obtain a new real-time image; Establish a CNN model, and use historical images and added category labels to train the CNN model to obtain a trained CNN model; input new real-time images into the trained CNN model to obtain prediction results; Result judgment: judge whether the prediction result meets the expectation. If so, execute the step of obtaining the response; if not, output an alarm signal; Get response: Apply low-frequency vibration signal to the monitored target and get response signal; Obtain feedback: Apply real-time high-frequency ultrasonic signals to the monitoring target to obtain real-time feedback signals; Perform spectrum analysis on the response signal to obtain modal parameters; extract signal characteristics of the real-time feedback signal; Parameter judgment: judge whether the modal parameter belongs to the preset parameter range. If yes, execute the step of obtaining feedback; if no, output an alarm signal; Feature judgment: Determine whether the signal characteristics meet expectations. If so, execute the step of obtaining a response after a preset interval; if not, output an alarm signal.

2. The welding defect monitoring method based on vibroacoustics according to claim 1, characterized in that: The calculation model of the difference degree is as follows: ； Where r represents the degree of difference; A is the number of grayscale values ​​in the first data set; B is the number of grayscale values ​​in the second data set; is the average value of all gray values ​​in the first data set; is the average value of all gray values ​​in the second data set.

3. The welding defect monitoring method based on vibroacoustics according to claim 1, before executing the step of determining the seed point, the method further comprises: Obtain the remaining fourth data in the fourth data set, recorded as sixth data; Acquire sixth data capable of forming a closed figure, recorded as seventh data; retain the seventh data in the fourth data set, and delete the remaining fourth data; In the step of determining the seed point, any pixel point corresponding to the seventh data is used as the seed point.

4. The welding defect monitoring method based on vibroacoustics according to any one of claims 1 to 3, characterized in that: After executing the step of feature determination and before outputting the alarm signal, the method further includes: Collect historical feedback signals of monitoring targets; Add a type label to the historical feedback signal, and record the historical feedback signal after the type label is added as the first signal; Modeling and training: Establish an RNN model, use the first signal to train the RNN model, and obtain the trained RNN model; Recognition: Input the real-time feedback signal into the trained RNN model to obtain the predicted type label; Label judgment: Determine whether the predicted type label is an abnormal label. If so, output an alarm signal; if not, execute the step of obtaining a response.

5. The welding defect monitoring method based on vibroacoustics according to claim 4 is characterized in that: After executing the modeling and training steps and before executing the recognition step, it also includes: Collecting the historical high-frequency ultrasonic signal corresponding to the historical feedback signal, and the difference between the time of collecting the historical feedback signal and the time of collecting the historical high-frequency ultrasonic signal; In the trained RNN model, multiple hidden layers are added to obtain an expanded RNN model, the added hidden layer in the expanded RNN model is recorded as the first hidden layer, and the original hidden layer in the expanded RNN model is recorded as the second hidden layer; The second hidden layer is fixed, and the historical high-frequency ultrasonic signal and the difference are used to train the first hidden layer to obtain a retrained RNN model; The retrained RNN model is used as a new trained RNN model and the recognition steps are performed; In the recognition step, the difference between the time of collecting the real-time feedback signal and the time of collecting the real-time high-frequency ultrasonic signal is also input into the new trained RNN model.

6. A welding defect monitoring system based on vibroacoustics, the system being applicable to the method according to any one of claims 1 to 5, characterized in that: include: The data acquisition module includes a first acquisition unit and a second acquisition unit; The first acquisition unit is used to apply a low-frequency vibration signal to the monitoring target and obtain a response signal; The second acquisition unit is used to apply a real-time high-frequency ultrasonic signal to the monitoring target to obtain a real-time feedback signal; A data processing module, comprising a first processing unit and a second processing unit; A first processing unit is used to perform spectrum analysis on the response signal to obtain modal parameters; A second processing unit, used for extracting signal features of the real-time feedback signal; Defect judgment module, including parameter judgment unit and feature judgment unit; A parameter judgment unit, used to judge whether the modal parameter belongs to a preset parameter range, and if so, trigger the second acquisition unit; otherwise, trigger the output module; A feature judgment unit is used to judge whether the signal feature meets expectations. If so, the first acquisition unit is triggered after a preset time interval; otherwise, the output module is triggered; Output module, used to output alarm signals.

Images