Welding detection method and application thereof in welding monitoring of LNG (Liquefied Natural Gas) storage tank

Abstract

The invention discloses a welding detection method and application thereof in welding monitoring of an LNG (liquefied natural gas) storage tank. According to the scheme, 3D point cloud data, surface image data and cross-sectional image data acquired from a welding seam by combining a 3D laser scanner, an industrial camera and an ultrasonic phased array probe are subjected to multi-modal data fusion; the fusion data is input into a trained defect recognition model, so that the weld joint can have a more perfect data source in quality detection classification, and a reliable data basis is provided for weld joint detection; besides, according to the scheme, a welding seam model based on digital twinning is constructed on the basis of fusion data, in this way, when the welding seam has defects caused by abnormal welding, the welding seam model constructed by the digital twinning model can be used for positioning the defects and outputting local welding seam model data with the defects, and meanwhile, by combining a detection result, the defect detection accuracy is improved. And more visual and reliable detection feedback can be provided for background management personnel.

Description

Technical Field

[0001] The present invention relates to the fields of welding technology, welding detection and LNG storage tank safety monitoring technology, and in particular to a welding detection method and its application in LNG storage tank welding monitoring. Background Art

[0002] LNG (Liquefied Natural Gas) tanks are special cryogenic pressure vessels, and their manufacturing and maintenance processes have extremely strict requirements on welding quality. The quality of the weld directly affects the safe operation and service life of the tank, so high-precision and comprehensive inspection and evaluation of the welding area of ​​LNG tanks has become a core technical requirement of the industry.

[0003] Traditional weld inspection methods mainly include manual visual inspection, traditional non-destructive testing (such as ultrasonic testing, radiographic testing, magnetic particle testing, penetrant testing, etc.) and some automated testing methods based on simple single sensors. These methods have the following shortcomings in practical applications: 1. Limitations of manual detection: Relying on experienced professionals to visually inspect the weld surface or use simple tools to evaluate is not only time-consuming, inefficient, and easily affected by human factors, but also difficult to achieve large-scale, continuous quantitative testing. In addition, manual testing in high altitude, low temperature environments or dangerous working conditions poses a great safety risk.

[0004] 2. Disadvantages of single sensor detection: Although traditional non-destructive testing instruments (such as ordinary ultrasonic probes or radiographic imaging equipment) can detect internal defects, they often lack the ability to accurately measure surface geometric features. At the same time, a single data source is difficult to effectively connect surface morphology and internal structure information, and it is impossible to form a systematic judgment on the overall quality of the weld.

[0005] 3. Low data utilization and lack of intelligent analysis: Most existing inspection systems are independent devices, with data in an isolated state, lacking effective means of cross-sensor and cross-data source integration. It is impossible to invert key process parameters such as weld shape, trajectory, and cross-sectional profile from continuous inspection data, and it is difficult to achieve intelligent analysis and trend prediction based on a large amount of inspection data.

[0006] With the rapid development of sensor technology and data processing capabilities, multi-sensor fusion detection and intelligent analysis have become a trend. By combining surface data acquisition methods (such as 3D laser scanning and structured light imaging) with internal non-destructive testing methods (such as phased array ultrasound, TOFD, and digital radiography), comprehensive data on the weld surface and interior can be obtained in the same coordinate system. At the same time, with the help of deep learning, pattern recognition, and data fusion algorithms, not only can surface flaws and internal defects be automatically identified and classified, but multiple frames of continuous data can also be spliced ​​and inverted to accurately present welding trajectories, cross-sectional profiles, and shape features, thereby providing data support for welding process quality control and process optimization.

[0007] However, to achieve the above-mentioned multi-level, multi-modal data fusion and intelligent detection, it is necessary to solve many technical problems: including sensor calibration and registration, coordinate transformation and error compensation, training and deployment of deep learning models, high-precision fusion of multi-modal data in a unified coordinate system, and trajectory inversion and quantitative defect assessment through complex algorithms. The existing technology still needs to be deepened and improved in the research of these issues, and there is also room for improvement in field operability, environmental adaptability and real-time data processing.

[0008] In summary, the industry urgently needs a method and system that can perform comprehensive and high-precision inspection of the weld area of ​​LNG storage tanks from both surface and internal dimensions to overcome the limitations of traditional inspection methods, and through data fusion and intelligent analysis technology, realize the closed loop of welding quality evaluation and process parameter reverse deduction, thereby greatly improving the automation and intelligence level of LNG storage tank welding quality control and maintenance. Summary of the invention

[0009] In view of this, the object of the present invention is to propose a welding detection method with reliable implementation, flexible application and good reliability of detection results, and its application in LNG storage tank welding monitoring.

[0010] In order to achieve the above technical objectives, the present invention adopts the following technical solutions: A welding detection method, applied to LNG storage tanks, comprising: S01. Continuously scan the weld surface of the LNG storage tank by using a 3D laser scanner and an industrial camera to generate 3D point cloud data and surface image data respectively; S02, scanning the inside of the weld with multi-angle sound beams using an ultrasonic phased array probe to generate tomographic image data for characterizing the internal structure of the weld; S03, spatially registering the 3D point cloud data, the surface image data, and the tomographic image data in a unified coordinate system, and then fusing the multimodal data to obtain fused data; S04. Input the fused data into a trained defect recognition model to determine whether the weld has internal defects and / or external defects, and generate a detection result.

[0011] As a possible implementation mode, further, the 3D laser scanner, industrial camera, and ultrasonic phased array probe described in this solution are used to scan the weld through a mobile auxiliary tooling, wherein the mobile auxiliary tooling includes a wall climbing robot and / or a movable robot arm; The 3D point cloud data covers the corresponding weld height, width, and surface defect characteristics; The internal defects include one or more of internal cracks, internal pores, slag inclusions, and lack of fusion; The external defects include one or more of surface cracks, surface pores, weld bumps, undercuts, and spatters.

[0012] As a better implementation option, preferably, this solution S05 reconstructs the welding trajectory based on the fused data, and parameterizes the weld section and contour to construct a weld model based on digital twin.

[0013] As a better implementation option, preferably, this solution S06 obtains the test results. When it points to a defect in the weld, the position of the defect on the weld is located according to the test results, and then the area where the defect is located is located from the weld model based on the digital twin, and it is intercepted and output together with the test results.

[0014] As a preferred implementation option, preferably, this solution S03 includes: Establish a global coordinate system, denoted as W, and use it as the world coordinate system for the entire LNG tank inspection environment; The main body coordinate system of the mobile auxiliary tooling that carries a 3D laser scanner, an industrial camera, and an ultrasonic phased array probe for mobile scanning of welds is denoted as R, which is the coordinate system of the mobile auxiliary tooling during motion control; the coordinate system corresponding to the 3D laser scanner is denoted as L, the coordinate system of the industrial camera is denoted as C, and the coordinate system of the ultrasonic phased array probe is denoted as U; The 3D point cloud data generated by the 3D laser scanner is recorded in the form of a data set, which is defined as ,in, is the point cloud coordinate in the coordinate system L; The image sequence obtained by the industrial camera is recorded as a data set, which is defined as , where the pixel coordinates of each image are Satisfy the imaging model; The 3D point cloud data generated by the 3D laser scanner is converted from the coordinate system L to the global coordinate system W, which is defined as follows: in, is the point cloud coordinates of the 3D point cloud data in the global coordinate system W; is the point cloud coordinates of the 3D point cloud data in the coordinate system L; , which is the rigid transformation matrix from the coordinate system L where the 3D laser scanner is located to the coordinate system R where the mobile auxiliary tooling is located, , which is the rigid transformation matrix from the coordinate system R where the mobile auxiliary tooling is located to the global coordinate system W; After conversion, the global coordinate distribution of the weld surface point cloud data in the global coordinate system W is obtained, which is expressed as: The image captured by the industrial camera is fused with the point cloud captured by the 3D laser scanner. Assume that the external calibration parameters of the industrial camera and the 3D laser scanner are known, that is, It is known that the 3D point cloud data is projected onto the image plane, which is defined as: in, For the i The 3D point cloud data is mapped to the point cloud coordinates in the coordinate system C, which is in the form of ; For the i The point cloud coordinates of a 3D point cloud data in the coordinate system L are in the form of ; is the rigid transformation matrix from the coordinate system R where the mobile auxiliary tooling is located to the coordinate system C; , which is the rigid transformation matrix from the coordinate system L where the 3D laser scanner is located to the coordinate system R where the mobile auxiliary tooling is located; Using the intrinsic matrix of industrial cameras K , projecting the 3D point cloud data into a 2D plane image, which is defined as follows: in, It is the pixel coordinates of the 3D point cloud data on the corresponding image of the industrial camera; for Depth K is the intrinsic parameter matrix of the industrial camera, which is used to map the 3D point cloud data to the image plane of the image data obtained by the industrial camera; By matching the surface point cloud features corresponding to the 3D point cloud data with the image texture features of the surface image data acquired by the industrial camera, the surface defects of the weld can be preliminarily located and characterized. The weld surface section can be extracted from the point cloud data by slicing method. Assuming that a center line is defined along the weld direction, the section perpendicular to the center line is fitted, including: Define Section Plane , the equation in the global coordinate system W can be defined as: ,in, is the cross-section normal vector, is the reference point of the section plane; Point cloud data In the middle A quadratic curve is fitted to nearby points, which is defined as follows: in, is the coordinate in the local coordinate system of the cross section, is the parameter to be fitted, which is solved by the least squares method and is defined as follows: The fitted parameters are used to calculate the specification parameters of the weld section, including the reinforcement height and width.

[0015] As a preferred implementation option, preferably, this solution S03 also includes: The tomographic image data acquired by the ultrasonic phased array probe is regarded as a volume data set, which contains the ultrasonic signal intensity of voxels or grid points. , is the scanning direction index; The tomographic image data acquired by the ultrasonic phased array probe is obtained in the coordinate system U, which is defined as: The rigid transformation matrix from the coordinate system R of the mobile auxiliary tooling to the coordinate system U of the ultrasonic phased array probe is obtained by calibration. , then the tomographic image data acquired by ultrasound acquisition is converted to the global coordinate system W, which is defined as follows: in, is the coordinate of the tomographic image data obtained by ultrasound acquisition in the global coordinate system W; The tomographic image data in the coordinate system U acquired by the ultrasonic phased array probe; is the rigid transformation matrix from the coordinate system R where the mobile auxiliary tooling is located to the global coordinate system W; is the rigid transformation matrix from the coordinate system U where the ultrasonic phased array probe is located to the coordinate system R where the mobile auxiliary tooling is located; Among them, the ultrasonic signal intensity distribution The coordinates of the tomographic image data obtained by ultrasound acquisition in the global coordinate system W One-to-one correspondence, the data in the global coordinate system can be expressed as .

[0016] As a preferred implementation option, preferably, in the present solution S03, the tomographic image data is spatially registered in a unified coordinate system, and then when multimodal data fusion is performed, the tomographic image data in the global coordinate system is also spatially registered using a trained deep learning model according to the ultrasonic signal characteristics of the tomographic image data. Defect location is performed, wherein the located defect appears as a signal anomaly in a specific area inside the weld, which is located by defining a threshold or a probability model, which is defined as follows: in, is a deep learning function, is a training parameter, if Greater than threshold , then it is judged that there is a defect point; By clustering in the ultrasonic phased array probe scanning area, the defect volume can be obtained and recorded as , which corresponds to a set of coordinates and feature points.

[0017] As a preferred implementation option, preferably, this solution S03 also includes: Assuming that the reference has been established at the start point, end point or preset specific reference point of the welding segment of the weld, if registration is still required, the surface points and ultrasonic internal reference points with corresponding features are selected for iterative closest point ICP or feature registration, where the ICP objective function is defined as follows: in, , which is the source point cloud data of the weld surface The i The three-dimensional coordinates of the points, Indicates the target point cloud The corresponding matching points, and , which are respectively represented as a rotation matrix and a translation vector, and the internal data is further aligned with the surface data through the rigid body transformation to the final minimum error through ICP iteration.

[0018] As a preferred implementation option, preferably, in the present solution S03, after processing, the 3D point cloud data collected by the 3D laser scanner , surface image data collected by industrial cameras Tomographic image data acquired by ultrasonic phased array probe The multimodal fusion data obtained after fusion is defined as follows: In order to facilitate defect classification, a deep learning classification model is constructed, which is defined as follows: in, It is a deep learning model based on CNN or Transformer structure. is its parameter, the input is the fused multimodal fusion data, and the output is Defect type label.

[0019] As a preferred implementation option, preferably, this solution S05 includes: Acquire multimodal fusion data, extract the characteristic points of the weld area in the weld surface point cloud data, virtually connect the highest points of the weld into a line, and define a set of reference sections along the welding direction ; Find the highest point of the weld on each section , that is, find the maximum value point in the quadratic curve fitting; obtain a set of sequence points, which are defined as follows: Using Kalman filter algorithm to process data Perform smoothing and filtering, which includes the following: The state prediction equation is established and is defined as follows: in, for The estimated state value at time is the estimated value of the state after the previous update, which represents The optimal estimate of the state at the moment, A is the state transfer matrix, B is the control input matrix, is the control input vector; The covariance prediction equation is established and is defined as follows: in, for The state estimation error covariance matrix after updating at each moment, is the state estimation error covariance matrix updated at the last moment, which describes The uncertainty of; A is the state transfer matrix; is the process noise covariance matrix; The measurement update equation is as follows: in, is the Kalman gain matrix; is the measurement matrix, is the state prediction error covariance matrix; is the measurement noise covariance matrix; The state update equation is as follows: is the updated trajectory point, is the observation point, which represents the highest point of the weld section. is the state transfer matrix; is the measurement matrix; After Kalman filtering, a smooth weld centerline is obtained , set as a smooth trajectory; Using cross-sectional fitting parameters With smooth trajectory The data is collected to build a weld cross-sectional morphology database, and then a weld model based on digital twin is built simultaneously.

[0020] Based on the above, this solution also provides a LNG tank welding monitoring method, which includes: A01, respond to the feedback information of the completion of LNG tank welding; A02. Execute the welding detection method described above.

[0021] Based on the above, this solution also provides an LNG tank welding monitoring system, which includes: 3D laser scanner, used to continuously scan the weld surface of LNG storage tanks and generate 3D point cloud data; Industrial cameras are used to continuously scan the weld surface of LNG tanks and generate surface image data; Ultrasonic phased array probe, used to scan the inside of the weld with multi-angle sound beams and generate tomographic image data for characterizing the internal structure of the weld; Mobile auxiliary tooling, used to carry 3D laser scanners, industrial cameras and ultrasonic phased array probes to collect data on welds on LNG tanks; The data fusion module is used to spatially register the 3D point cloud data, surface image data and tomographic image data in a unified coordinate system, and then perform multimodal data fusion to obtain fused data; A data judgment module is used to input the fused data into a trained defect recognition model to judge whether the weld has internal defects and / or external defects and generate a detection result; The digital twin module is used to reconstruct the welding trajectory based on the fused data, parameterize the weld section and profile, and build a weld model based on the digital twin; The result output module is used to obtain the detection results. When it points to defects in the weld, the position of the defect on the weld is located according to the detection results, and then the area where the defect is located is located from the weld model based on the digital twin, and it is intercepted and output together with the detection results.

[0022] By adopting the above-mentioned technical scheme, the present invention has the following beneficial effects compared with the prior art: this scheme collects 3D point cloud data, surface image data and tomographic image data of the weld by combining a 3D laser scanner, an industrial camera and an ultrasonic phased array probe, and after multimodal data fusion, the fused data is input into the trained defect recognition model, so that the weld can have a more complete data source in the quality inspection classification, providing a reliable data basis for the inspection of the weld; in addition, this scheme also constructs a weld model based on a digital twin based on the fused data. In this way, when there are defects in the weld caused by welding abnormalities, the weld model constructed by the digital twin model can be used to locate the defects and output the local weld model data with defects. At the same time, combined with the inspection results, more intuitive and reliable inspection feedback can be provided to the back-end management personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 This is a schematic diagram of one implementation process of the detection method of this scheme; Figure 2 It is another implementation flow diagram of the detection method of this scheme; Figure 3 This is a brief connection diagram of the unit modules of the monitoring system of this scheme. DETAILED DESCRIPTION

[0025] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0026] like Figure 1 As shown, this embodiment provides a welding detection method, which is applied to LNG storage tanks and includes: S01. Continuously scan the weld surface of the LNG storage tank by using a 3D laser scanner and an industrial camera to generate 3D point cloud data and surface image data respectively; S02, scanning the inside of the weld with multi-angle sound beams using an ultrasonic phased array probe to generate tomographic image data for characterizing the internal structure of the weld; S03, spatially registering the 3D point cloud data, the surface image data, and the tomographic image data in a unified coordinate system, and then fusing the multimodal data to obtain fused data; S04. Input the fused data into a trained defect recognition model to determine whether the weld has internal defects and / or external defects, and generate a detection result.

[0027] This solution uses a 3D laser scanner, an industrial camera, and an ultrasonic phased array probe to collect 3D point cloud data, surface image data, and tomographic image data from welds. After multimodal data fusion, the fused data is input into a trained defect recognition model. This allows welds to have a more complete data source in quality inspection and classification, providing a reliable data basis for weld inspection.

[0028] Among them, as a possible implementation mode, further, the 3D laser scanner, industrial camera, and ultrasonic phased array probe described in this solution are used to scan the weld through a mobile auxiliary tooling, wherein the mobile auxiliary tooling includes a wall climbing robot and / or a movable robot arm; The 3D point cloud data covers the corresponding weld height, width, and surface defect characteristics; The internal defects include one or more of internal cracks, internal pores, slag inclusions, and lack of fusion; The external defects include one or more of surface cracks, surface pores, weld bumps, undercuts, and spatters.

[0029] Combination Figure 2 As shown, in order to make a better and more intuitive judgment and analysis of the weld, as a better implementation option, preferably, this scheme S05, based on the fusion data, reconstructs the path of the welding trajectory, and parameterizes the weld section and contour to construct a weld model based on digital twin.

[0030] On the basis of the above, as a better implementation option, preferably, this solution S06 obtains the test results. When it points to a defect in the weld, the position of the defect on the weld is located according to the test results, and then the area where the defect is located is located from the weld model based on the digital twin, and it is intercepted and output together with the test results.

[0031] This solution cleverly constructs a digital twin weld model based on fused data. In this way, when there are defects in the weld due to welding abnormalities, the weld model constructed by the digital twin model can be used to locate the defects and output the local weld model data with defects. At the same time, combined with the detection results, it can provide back-end management personnel with more intuitive and reliable detection feedback.

[0032] Since there are certain coordinate differences in data collection between 3D laser scanners, industrial cameras, and ultrasonic phased array probes, in order to conveniently and reliably perform correlation analysis on the data, as a preferred implementation option, preferably, this solution S03 includes: Establish a global coordinate system, denoted as W, and use it as the world coordinate system for the entire LNG tank inspection environment; The main body coordinate system of the mobile auxiliary tooling that carries a 3D laser scanner, an industrial camera, and an ultrasonic phased array probe for mobile scanning of welds is denoted as R, which is the coordinate system of the mobile auxiliary tooling during motion control; the coordinate system corresponding to the 3D laser scanner is denoted as L, the coordinate system of the industrial camera is denoted as C, and the coordinate system of the ultrasonic phased array probe is denoted as U; The 3D point cloud data generated by the 3D laser scanner is recorded in the form of a data set, which is defined as ,in, is the point cloud coordinate in the coordinate system L; The image sequence obtained by the industrial camera is recorded as a data set, which is defined as , where the pixel coordinates of each image are Satisfy the imaging model; The 3D point cloud data generated by the 3D laser scanner is converted from the coordinate system L to the global coordinate system W, which is defined as follows: in, is the point cloud coordinates of the 3D point cloud data in the global coordinate system W; is the point cloud coordinates of the 3D point cloud data in the coordinate system L; , which is the rigid transformation matrix from the coordinate system L where the 3D laser scanner is located to the coordinate system R where the mobile auxiliary tooling is located, , which is the rigid transformation matrix from the coordinate system R where the mobile auxiliary tooling is located to the global coordinate system W; After conversion, the global coordinate distribution of the weld surface point cloud data in the global coordinate system W is obtained, which is expressed as: The image captured by the industrial camera is fused with the point cloud captured by the 3D laser scanner. Assume that the external calibration parameters of the industrial camera and the 3D laser scanner are known, that is, It is known that the 3D point cloud data is projected onto the image plane, which is defined as: in, For the i The 3D point cloud data is mapped to the point cloud coordinates in the coordinate system C, which is in the form of ; For the i The point cloud coordinates of a 3D point cloud data in the coordinate system L are in the form of ; is the rigid transformation matrix from the coordinate system R where the mobile auxiliary tooling is located to the coordinate system C; , which is the rigid transformation matrix from the coordinate system L where the 3D laser scanner is located to the coordinate system R where the mobile auxiliary tooling is located; Using the intrinsic matrix of industrial cameras K , projecting the 3D point cloud data into a 2D plane image, which is defined as follows: in, It is the pixel coordinates of the 3D point cloud data on the corresponding image of the industrial camera; for Depth K is the intrinsic parameter matrix of the industrial camera, which is used to map the 3D point cloud data to the image plane of the image data obtained by the industrial camera; By matching the surface point cloud features corresponding to the 3D point cloud data with the image texture features of the surface image data acquired by the industrial camera, the surface defects of the weld can be preliminarily located and characterized. The weld surface section can be extracted from the point cloud data by slicing method. Assuming that a center line is defined along the weld direction, the section perpendicular to the center line is fitted, including: Define Section Plane , the equation in the global coordinate system W can be defined as: ,in, is the cross-section normal vector, is the reference point of the section plane; Point cloud data In the middle A quadratic curve is fitted to nearby points, which is defined as follows: in, is the coordinate in the local coordinate system of the cross section, is the parameter to be fitted, which is solved by the least squares method and is defined as follows: The fitted parameters are used to calculate the specification parameters of the weld section, including the reinforcement height and width.

[0033] As a preferred implementation option, preferably, this solution S03 also includes: The tomographic image data acquired by the ultrasonic phased array probe is regarded as a volume data set, which contains the ultrasonic signal intensity of voxels or grid points. , is the scanning direction index; The tomographic image data acquired by the ultrasonic phased array probe is obtained in the coordinate system U, which is defined as: The rigid transformation matrix from the coordinate system R of the mobile auxiliary tooling to the coordinate system U of the ultrasonic phased array probe is obtained by calibration. , then the tomographic image data acquired by ultrasound acquisition is converted to the global coordinate system W, which is defined as follows: in, is the coordinate of the tomographic image data obtained by ultrasound acquisition in the global coordinate system W; The tomographic image data in the coordinate system U acquired by the ultrasonic phased array probe; is the rigid transformation matrix from the coordinate system R where the mobile auxiliary tooling is located to the global coordinate system W; is the rigid transformation matrix from the coordinate system U where the ultrasonic phased array probe is located to the coordinate system R where the mobile auxiliary tooling is located; Among them, the ultrasonic signal intensity distribution The coordinates of the tomographic image data obtained by ultrasound acquisition in the global coordinate system W One-to-one correspondence, the data in the global coordinate system can be expressed as .

[0034] In addition to the above, in the present solution S03, the tomographic image data is spatially registered in a unified coordinate system, and then multimodal data fusion is performed. The tomographic image data in the global coordinate system is also spatially registered using a trained deep learning model according to the ultrasonic signal characteristics of the tomographic image data. Defect location is performed, wherein the located defect appears as a signal anomaly in a specific area inside the weld, which is located by defining a threshold or a probability model, which is defined as follows: in, is a deep learning function, is a training parameter, if Greater than threshold , then it is judged that there is a defect point; By clustering in the ultrasonic phased array probe scanning area, the defect volume can be obtained and recorded as , which corresponds to a set of coordinates and feature points.

[0035] This solution maps the data collected by 3D laser scanners, industrial cameras, and ultrasonic phased array probes into a unified coordinate system by constructing a global coordinate system W, so that the data joint analysis has a more reliable correlation basis.

[0036] As for the reliability of the unified coordinates of the data, as a better implementation option, preferably, the solution S03 further includes: Assuming that the reference has been established at the start point, end point or preset specific reference point of the welding segment of the weld, if registration is still required, the surface points and ultrasonic internal reference points with corresponding features are selected for iterative closest point ICP or feature registration, where the ICP objective function is defined as follows: in, , which is the source point cloud data of the weld surface The i The three-dimensional coordinates of the points, Indicates the target point cloud The corresponding matching points, and , which are respectively represented as a rotation matrix and a translation vector, and the internal data is further aligned with the surface data through the rigid body transformation to the final minimum error through ICP iteration.

[0037] In order to improve the convenience and speed of data analysis, this solution can provide some preliminary analysis during data fusion as a better implementation option. Preferably, in this solution S03, after processing, the 3D point cloud data collected by the 3D laser scanner , surface image data collected by industrial cameras Tomographic image data acquired by ultrasonic phased array probe The multimodal fusion data obtained after fusion is defined as follows: In order to facilitate defect classification, a deep learning classification model is constructed, which is defined as follows: in, It is a deep learning model based on CNN or Transformer structure. is its parameter, the input is the fused multimodal fusion data, and the output is Defect type label.

[0038] In this solution S04, the fused data is input into a trained defect recognition model. The defect recognition model can be a detection neural network, which, after being trained to convergence by the corresponding data set, can be used to determine whether the weld has internal defects and / or external defects and generate detection results.

[0039] Since the training mechanism of the detection neural network is a prior art, it will not be described in detail here.

[0040] This solution is a better implementation option for the inversion of the weld model. Preferably, this solution S05 includes: Acquire multimodal fusion data, extract the characteristic points of the weld area in the weld surface point cloud data, virtually connect the highest points of the weld into a line, and define a set of reference sections along the welding direction ; Find the highest point of the weld on each section , that is, find the maximum value point in the quadratic curve fitting; obtain a set of sequence points, which are defined as follows: Using Kalman filter algorithm to process data Perform smoothing and filtering, which includes the following: The state prediction equation is established and is defined as follows: in, for The estimated state value at time is the estimated value of the state after the previous update, which represents The optimal estimate of the state at the moment, A is the state transfer matrix, B is the control input matrix, is the control input vector; The covariance prediction equation is established and is defined as follows: in, for The state estimation error covariance matrix after updating at each moment, is the state estimation error covariance matrix updated at the last moment, which describes The uncertainty of; A is the state transfer matrix; is the process noise covariance matrix; The measurement update equation is as follows: in, is the Kalman gain matrix; is the measurement matrix, is the state prediction error covariance matrix; is the measurement noise covariance matrix; The state update equation is as follows: is the updated trajectory point, is the observation point, which represents the highest point of the weld section. is the state transfer matrix; is the measurement matrix; After Kalman filtering, a smooth weld centerline is obtained , set as a smooth trajectory; Using cross-sectional fitting parameters With smooth trajectory The data is collected to build a weld cross-sectional morphology database, and then a weld model based on digital twin is built simultaneously.

[0041] Based on the above, this solution also provides a LNG tank welding monitoring method, which includes: A01, respond to the feedback information of the completion of LNG tank welding; A02. Execute the welding detection method described above.

[0042] Combination Figure 3 As shown, based on the above, this solution also provides an LNG tank welding monitoring system, which includes: 3D laser scanner, used to continuously scan the weld surface of LNG storage tanks and generate 3D point cloud data; Industrial cameras are used to continuously scan the weld surface of LNG tanks and generate surface image data; Ultrasonic phased array probe, used to scan the inside of the weld with multi-angle sound beams and generate tomographic image data for characterizing the internal structure of the weld; Mobile auxiliary tooling, used to carry 3D laser scanners, industrial cameras and ultrasonic phased array probes to collect data on welds on LNG tanks; The data fusion module is used to spatially register the 3D point cloud data, surface image data and tomographic image data in a unified coordinate system, and then perform multimodal data fusion to obtain fused data; A data judgment module is used to input the fused data into a trained defect recognition model to judge whether the weld has internal defects and / or external defects and generate a detection result; The digital twin module is used to reconstruct the welding trajectory based on the fused data, parameterize the weld section and profile, and build a weld model based on the digital twin; The result output module is used to obtain the detection results. When it points to defects in the weld, the position of the defect on the weld is located according to the detection results, and then the area where the defect is located is located from the weld model based on the digital twin, and it is intercepted and output together with the detection results.

[0043] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0044] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0045] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A welding detection method, applied to LNG storage tanks, characterized in that: It includes: S01. Continuously scan the weld surface of the LNG storage tank by using a 3D laser scanner and an industrial camera to generate 3D point cloud data and surface image data respectively; S02, scanning the inside of the weld with multi-angle sound beams using an ultrasonic phased array probe to generate tomographic image data for characterizing the internal structure of the weld; S03, spatially registering the 3D point cloud data, the surface image data, and the tomographic image data in a unified coordinate system, and then fusing the multimodal data to obtain fused data; S04. Input the fused data into a trained defect recognition model to determine whether the weld has internal defects and / or external defects, and generate a detection result.

2. The welding detection method according to claim 1, characterized in that: The 3D laser scanner, industrial camera, and ultrasonic phased array probe are used to scan the weld seam through a mobile auxiliary tooling, wherein the mobile auxiliary tooling includes a wall climbing robot and / or a movable robot arm; The 3D point cloud data covers the corresponding weld height, width, and surface defect characteristics; The internal defects include one or more of internal cracks, internal pores, slag inclusions, and lack of fusion; The external defects include one or more of surface cracks, surface pores, weld bumps, undercuts, and spatters.

3. The welding detection method according to claim 2, characterized in that: S05. Based on the fused data, the welding trajectory is reconstructed, and the weld section and profile are parameterized to build a weld model based on digital twins.

4. The welding detection method according to claim 3, characterized in that: S06. Obtain the test results. When it points to defects in the weld, locate the position of the defect on the weld according to the test results, then locate the area where the defect is located from the weld model based on the digital twin, intercept it, and output it together with the test results.

5. The welding detection method according to any one of claims 3 to 4, characterized in that: S03 includes: Establish a global coordinate system, denoted as W, and use it as the world coordinate system for the entire LNG tank inspection environment; The main body coordinate system of the mobile auxiliary tooling that carries a 3D laser scanner, an industrial camera, and an ultrasonic phased array probe for mobile scanning of welds is denoted as R, which is the coordinate system of the mobile auxiliary tooling during motion control; the coordinate system corresponding to the 3D laser scanner is denoted as L, the coordinate system of the industrial camera is denoted as C, and the coordinate system of the ultrasonic phased array probe is denoted as U; The 3D point cloud data generated by the 3D laser scanner is recorded in the form of a data set, which is defined as ,in, is the point cloud coordinate in the coordinate system L; The image sequence obtained by the industrial camera is recorded as a data set, which is defined as , where the pixel coordinates of each image are Satisfy the imaging model; The 3D point cloud data generated by the 3D laser scanner is converted from the coordinate system L to the global coordinate system W, which is defined as follows: in, is the point cloud coordinates of the 3D point cloud data in the global coordinate system W; is the point cloud coordinates of the 3D point cloud data in the coordinate system L; , which is the rigid transformation matrix from the coordinate system L where the 3D laser scanner is located to the coordinate system R where the mobile auxiliary tooling is located, , which is the rigid transformation matrix from the coordinate system R where the mobile auxiliary tooling is located to the global coordinate system W; After conversion, the global coordinate distribution of the weld surface point cloud data in the global coordinate system W is obtained, which is expressed as: The image captured by the industrial camera is fused with the point cloud captured by the 3D laser scanner. Assume that the external calibration parameters of the industrial camera and the 3D laser scanner are known, that is, It is known that the 3D point cloud data is projected onto the image plane, which is defined as: in, For the i The 3D point cloud data is mapped to the point cloud coordinates in the coordinate system C, which is in the form of ; For the i The point cloud coordinates of a 3D point cloud data in the coordinate system L are in the form of ; is the rigid transformation matrix from the coordinate system R where the mobile auxiliary tooling is located to the coordinate system C; , which is the rigid transformation matrix from the coordinate system L where the 3D laser scanner is located to the coordinate system R where the mobile auxiliary tooling is located; Using the intrinsic matrix of industrial cameras K , projecting the 3D point cloud data into a 2D plane image, which is defined as follows: in, It is the pixel coordinates of the 3D point cloud data on the corresponding image of the industrial camera; for Depth K is the intrinsic parameter matrix of the industrial camera, which is used to map the 3D point cloud data to the image plane of the image data obtained by the industrial camera; By matching the surface point cloud features corresponding to the 3D point cloud data with the image texture features of the surface image data acquired by the industrial camera, the surface defects of the weld can be preliminarily located and characterized. The weld surface section can be extracted from the point cloud data by slicing method. Assuming that a center line is defined along the weld direction, the section perpendicular to the center line is fitted, including: Define Section Plane , the equation in the global coordinate system W can be defined as: ,in, is the cross-section normal vector, is the reference point of the section plane; Point cloud data In the middle A quadratic curve is fitted to nearby points, which is defined as follows: in, is the coordinate in the local coordinate system of the cross section, is the parameter to be fitted, which is solved by the least squares method and is defined as follows: The fitted parameters are used to calculate the specification parameters of the weld section, including the reinforcement height and width.

6. The welding detection method according to claim 5, characterized in that: S03 includes: The tomographic image data acquired by the ultrasonic phased array probe is regarded as a volume data set, which contains the ultrasonic signal intensity of voxels or grid points. , is the scanning direction index; The tomographic image data acquired by the ultrasonic phased array probe is obtained in the coordinate system U, which is defined as: The rigid transformation matrix from the coordinate system R of the mobile auxiliary tooling to the coordinate system U of the ultrasonic phased array probe is obtained by calibration. , then the tomographic image data acquired by ultrasound acquisition is converted to the global coordinate system W, which is defined as follows: in, is the coordinate of the tomographic image data obtained by ultrasound acquisition in the global coordinate system W; The tomographic image data in the coordinate system U acquired by the ultrasonic phased array probe; is the rigid transformation matrix from the coordinate system R where the mobile auxiliary tooling is located to the global coordinate system W; is the rigid transformation matrix from the coordinate system U where the ultrasonic phased array probe is located to the coordinate system R where the mobile auxiliary tooling is located; Among them, the ultrasonic signal intensity distribution The coordinates of the tomographic image data obtained by ultrasound acquisition in the global coordinate system W One-to-one correspondence, the data in the global coordinate system can be expressed as .

7. The welding detection method according to claim 6, characterized in that: In S03, the tomographic image data is spatially registered in a unified coordinate system, and then when multimodal data fusion is performed, the tomographic image data in the global coordinate system is fused using the trained deep learning model according to the ultrasonic signal characteristics of the tomographic image data. Defect location is performed, wherein the located defect appears as a signal anomaly in a specific area inside the weld, which is located by defining a threshold or a probability model, which is defined as follows: in, is a deep learning function, is a training parameter, if Greater than threshold , then it is judged that there is a defect point; By clustering in the ultrasonic phased array probe scanning area, the defect volume can be obtained and recorded as , which corresponds to a set of coordinates and feature points; S03 also includes: Assuming that a reference has been established at the start point, end point or preset specific reference point of the weld segment, if registration is still required, the surface points and ultrasonic internal reference points with corresponding features are selected for iterative closest point ICP or feature registration, where the ICP objective function is defined as follows: in, , which is the source point cloud data of the weld surface The i The three-dimensional coordinates of the points, Indicates the target point cloud The corresponding matching points, and , which are respectively represented as rotation matrices and translation vectors, and the internal data are further aligned with the surface data through the rigid body transformation to the final minimum error through ICP iteration; In S03, after processing, the 3D point cloud data collected by the 3D laser scanner , surface image data collected by industrial cameras Tomographic image data acquired by ultrasonic phased array probe The multimodal fusion data obtained after fusion is defined as follows: In order to facilitate defect classification, a deep learning classification model is constructed, which is defined as follows: in, It is a deep learning model based on CNN or Transformer structure. is its parameter, the input is the fused multimodal fusion data, and the output is Defect type label.

8. The welding detection method according to claim 7, characterized in that: S05 includes: Acquire multimodal fusion data, extract the characteristic points of the weld area in the weld surface point cloud data, virtually connect the highest points of the weld into a line, and define a set of reference sections along the welding direction ; Find the highest point of the weld on each section , that is, find the maximum value point in the quadratic curve fitting; obtain a set of sequence points, which are defined as follows: Using Kalman filter algorithm to process data Perform smoothing and filtering, which includes the following: The state prediction equation is established and is defined as follows: in, for The estimated state value at time is the estimated value of the state after the previous update, which represents The optimal estimate of the state at the moment, A is the state transfer matrix, B is the control input matrix, is the control input vector; The covariance prediction equation is established and is defined as follows: in, for k The state estimation error covariance matrix after updating at each moment, is the state estimation error covariance matrix updated at the last moment, which describes The uncertainty of; A is the state transfer matrix; is the process noise covariance matrix; The measurement update equation is as follows: in, is the Kalman gain matrix; is the measurement matrix, is the state prediction error covariance matrix; is the measurement noise covariance matrix; The state update equation is as follows: is the updated trajectory point, is the observation point, which represents the highest point of the weld section. is the state transfer matrix; is the measurement matrix; After Kalman filtering, a smooth weld centerline is obtained , set as a smooth trajectory; Using cross-sectional fitting parameters With smooth trajectory The data is collected to build a weld cross-sectional morphology database, and then a weld model based on digital twin is built simultaneously.

9. A method for monitoring welding of LNG storage tanks, characterized in that: It includes: A01, respond to the feedback information of the completion of LNG tank welding; A02. Execute the welding detection method described in any one of claims 1 to 8. 10.LNG tank welding monitoring system, characterized in that: It includes: 3D laser scanner, used to continuously scan the weld surface of LNG storage tanks and generate 3D point cloud data; Industrial cameras are used to continuously scan the weld surface of LNG tanks and generate surface image data; Ultrasonic phased array probe, used to scan the inside of the weld with multi-angle sound beams and generate tomographic image data for characterizing the internal structure of the weld; Mobile auxiliary tooling, used to carry 3D laser scanners, industrial cameras and ultrasonic phased array probes to collect data on welds on LNG tanks; The data fusion module is used to spatially register the 3D point cloud data, surface image data and tomographic image data in a unified coordinate system, and then perform multimodal data fusion to obtain fused data; A data judgment module is used to input the fused data into a trained defect recognition model to judge whether the weld has internal defects and / or external defects and generate a detection result; The digital twin module is used to reconstruct the welding trajectory based on the fused data, parameterize the weld section and profile, and build a weld model based on the digital twin; The result output module is used to obtain the detection results. When it points to defects in the weld, the position of the defect on the weld is located according to the detection results, and then the area where the defect is located is located from the weld model based on the digital twin, and it is intercepted and output together with the detection results.

Images