A method for weld defect detection based on robot-mounted vision

Abstract

This invention relates to a weld defect detection method based on a robot equipped with vision, which solves the technical problems of low efficiency, low accuracy, and low safety in manual quality inspection of welds in reactor pressure vessels by workers using handheld ultrasonic probes. The method first locates the weld trajectory using multiple digital stamps, then acquires images of the multiple digital stamps using machine vision and calculates the motion trajectory, and finally controls the movement of the robotic arm by combining force feedback impedance control to make the ultrasonic probe conform to the motion trajectory for detection.

Description

Technical Field

[0001] This invention relates to the field of weld defect detection technology for reactor pressure vessels, and more specifically, to a weld defect detection method based on a robot equipped with vision. Background Technology

[0002] In the field of nuclear power plant technology, the reactor pressure vessel is a crucial piece of equipment. Nuclear power plants use nuclear fuel as a heat source and generate electricity through controlled nuclear fission technology. The nuclear fuel is placed in the reactor core, which is located in the lower middle part of the reactor pressure vessel and supported by eight core support blocks. As the nuclear boiler of a nuclear power plant, the reactor pressure vessel is an indispensable piece of equipment during the plant's operation. It is installed and fixed within the nuclear power plant during construction and cannot be replaced throughout the entire power generation period. Furthermore, it is costly and large in size. The lifespan of the reactor pressure vessel directly determines whether the nuclear power unit can operate safely for a long period, and is crucial for the safe power generation and normal operation of the nuclear power plant. During its service life, the reactor pressure vessel is subjected to high-temperature, high-pressure, and high-intensity radiation from fast neutrons for extended periods, leading to radiation embrittlement. Its failure modes primarily include brittle fracture, creep, corrosion, fatigue, and excessive strength failure, posing the primary threat to the safe operation of the reactor pressure vessel. The reactor pressure vessel is welded together from various parts, so there are strict requirements for the quality of each weld on the nuclear reactor pressure vessel. Weld quality inspections must be carried out during the construction of the pressure vessel, before it is put into use, and after a period of use, until the nuclear power plant is shut down.

[0003] Typically, workers use handheld ultrasonic probes to inspect the welds on the reactor pressure vessel. Before and during service, the reactor pressure vessel is completely immersed in coolant, and its surface is highly radioactive, making it impossible for workers to approach it for extended periods. Furthermore, manual inspection using ultrasonic probes suffers from low efficiency, low safety, and low accuracy. Summary of the Invention

[0004] This invention aims to address the technical problems of low efficiency, low accuracy, and low safety in manual quality inspection of welds in reactor pressure vessels using handheld ultrasonic probes. It provides a weld defect detection method based on a robot equipped with vision.

[0005] This invention provides a method for weld defect detection based on robot-mounted vision, including a weld inspection robot comprising a wall-climbing robot, a global binocular camera, a robotic arm, and an ultrasonic probe. The global binocular camera is connected to the wall-climbing robot, the robotic arm is connected to the wall-climbing robot, and the ultrasonic probe is connected to the end of the robotic arm.

[0006] The method for detecting weld defects includes the following steps:

[0007] The first step is to stamp several consecutive numbers along the weld seam at certain intervals, with the numbers located on the center line of the weld seam.

[0008] The second step is to take pictures of all the digital stamps with a camera beforehand to obtain a two-dimensional image of each digital stamp. For each two-dimensional image of the digital stamp, the SIFT algorithm is used to extract all the feature points of the stamp number. From all the feature points, some feature points are selected as feature points TD. The coordinates of the feature points TD are calculated.

[0009] The third step involves the wall-climbing robot moving the weld inspection robot to several consecutive digit stamps. The global binocular camera captures images of the area to be inspected, and the captured images are sent to the controller. The controller identifies the stamp numbers in the images, calculates the position information of the stamp numbers, and establishes a world coordinate system with the center of the stamp numbers as the origin.

[0010] The fourth step involves extracting all feature points Td of the stamped digits obtained in the third step using the SIFT algorithm. Then, the feature points Td and TD are matched using a brute-force matching method to obtain the feature point Td corresponding to each feature point TD as a matching point pair, and the successfully matched feature points Td are retained. The coordinates of the feature point Td in the world coordinate system are calculated.

[0011] The fifth step is to use the PnP algorithm to solve the mapping relationship between feature point TD and feature point Td based on the matching point pairs;

[0012] The sixth step involves solving the mapping relationship using the PnP algorithm to obtain the transformation matrix from the world coordinate system to the robot arm base coordinate system.

[0013] Step 7: Establish trajectory information in the world coordinate system, using the centers of all the stamped numbers obtained in step 3. The trajectory information meets the following conditions:

[0014]

[0015] Where R is the radius of the trajectory, and h is the height of the trajectory;

[0016] By transforming the matrix Calculate trajectory information Trajectory information in the robot arm base coordinate system:

[0017] In trajectory information The resulting trajectory is obtained by sampling multiple coordinate points at certain intervals, and these multiple coordinate points form trajectory information. Track information The rotation angles of each joint of the robotic arm are obtained by inverse kinematics, and then the motion trajectory of the robotic arm's end effector is obtained.

[0018] The eighth step involves controlling the robotic arm's movements. The ultrasonic probe at the end of the robotic arm moves along the motion trajectory and performs defect detection.

[0019] Preferably, in the second step, a subset of feature points are selected from all feature points through the following process: calculating the difference between the grayscale value of the pixel containing each feature point and the grayscale values ​​of the pixels above, below, left, and right to obtain the grayscale gradient value G. n Calculate the average gradient value of all feature points. Feature points with gradient values ​​greater than the average gradient value are retained. When a feature point in a region is retained multiple times, the multiple feature points in that region are merged. A circle of a certain diameter is used as the merging region, and the feature point closest to the center of the circle is selected and retained, finally obtaining the feature point TD.

[0020] Preferably, in the third step, the DBnet model and CRNN model are used to identify the stamped number. The image acquired by the global binocular camera is input to the DBnet model. The DBnet model outputs the pixel position of the detected target in the image and represents it with a detection box. Then, the image within the detection box output by the DBnet model is input to the CRNN module. The CRNN module outputs the detection result of the stamped number and identifies the stamped number.

[0021] Preferably, the weld trajectory is a circular weld trajectory.

[0022] Preferably, an arm-mounted camera is connected to the end of the robotic arm.

[0023] Preferably, the robotic arm is a six-axis robotic arm.

[0024] The present invention also provides a method for detecting weld defects based on a robot equipped with vision, including a weld detection robot comprising a wall-climbing robot, a global binocular camera, a robotic arm, an ultrasonic probe, and a six-dimensional force sensor. The global binocular camera is connected to the wall-climbing robot, the robotic arm is connected to the wall-climbing robot, the six-dimensional force sensor is connected to the end effector of the robotic arm, and the ultrasonic probe is connected to the six-dimensional force sensor.

[0025] The method for detecting weld defects includes the following steps:

[0026] The first step is to stamp several consecutive numbers along the weld seam at certain intervals, with the numbers located on the center line of the weld seam.

[0027] The second step is to take pictures of all the digital stamps with a camera beforehand to obtain a two-dimensional image of each digital stamp. For each two-dimensional image of the digital stamp, the SIFT algorithm is used to extract all the feature points of the stamp number. From all the feature points, some feature points are selected as feature points TD. The coordinates of the feature points TD are calculated.

[0028] The third step involves the wall-climbing robot moving the weld inspection robot to several consecutive digit stamps. The global binocular camera captures images of the area to be inspected, and the captured images are sent to the controller. The controller identifies the stamp numbers in the images, calculates the position information of the stamp numbers, and establishes a world coordinate system with the center of the stamp numbers as the origin.

[0029] The fourth step involves extracting all feature points Td of the stamped digits obtained in the third step using the SIFT algorithm. Then, the feature points Td and TD are matched using a brute-force matching method to obtain the feature point Td corresponding to each feature point TD as a matching point pair, and the successfully matched feature points Td are retained. The coordinates of the feature point Td in the world coordinate system are calculated.

[0030] The fifth step is to use the PnP algorithm to solve the mapping relationship between feature point TD and feature point Td based on the matching point pairs;

[0031] The sixth step involves solving the mapping relationship using the PnP algorithm to obtain the transformation matrix from the world coordinate system to the robot arm base coordinate system.

[0032] Step 7: Establish trajectory information in the world coordinate system, using the centers of all the stamped numbers obtained in step 3. The trajectory information meets the following conditions:

[0033]

[0034] Where R is the radius of the trajectory, and h is the height of the trajectory;

[0035] By transforming the matrix Calculate trajectory information Trajectory information in the robot arm base coordinate system:

[0036] In trajectory information The resulting trajectory is obtained by sampling multiple coordinate points at certain intervals, and these multiple coordinate points form trajectory information. Track information The rotation angles of each joint of the robotic arm are obtained by inverse kinematics, and then the motion trajectory of the robotic arm's end effector is obtained.

[0037] The eighth step involves using impedance control to control the robotic arm's movements. The ultrasonic probe at the end of the robotic arm moves along the motion trajectory and performs defect detection.

[0038] The formula for impedance control is:

[0039]

[0040] Wherein, the mass parameter M is the mass of the robotic arm itself, the damping parameter B is a set value, and the stiffness parameter K is a set value; X, It is the actual displacement, velocity, and acceleration of the ultrasonic probe, X. r , F represents the expected displacement, velocity, and acceleration of the ultrasonic probe; F is the actual pressure fed back by the six-dimensional force sensor; F r The expected pressure set;

[0041] Actual displacement X and velocity of the ultrasonic probe acceleration The angular velocity of the robotic arm is calculated by measuring the angles, angular velocities, and angular accelerations of each joint. The joint's photoelectric encoder detects the joint's angle θ, and the angular velocity is obtained by calculating the difference between angle θ and the calculated angle θ. angular velocity The angular acceleration is obtained by finite difference. Calculate the actual displacement X of the ultrasonic probe. Let be the transformation matrix from joint i-1 to joint i.

[0042]

[0043] Where, α i-1 ,θ i ,d i ,a i-1 The DH parameters for the robotic arm are established;

[0044] Ultrasonic probe speed Obtained through the Jacobian matrix of the robotic arm.

[0045] in,

[0046]

[0047] In J(θ), each term represents the partial derivative of the ultrasound probe position with respect to the corresponding joint angle.

[0048] acceleration of the ultrasonic probe By speed Taking the derivative, we get

[0049] Desired acceleration of the ultrasonic probe It is calculated using the following formula:

[0050]

[0051] Desired speed of ultrasonic probe It is calculated using the following formula:

[0052]

[0053] Desired displacement X of the ultrasonic probe r It is calculated using the following formula:

[0054]

[0055] X r , The angles, velocities, and accelerations of each joint of the robotic arm are calculated using the Jacobian matrix based on inverse kinematics. These angles, velocities, and accelerations are then input to the motion controller, which controls the robotic arm's movements to ensure the ultrasound probe moves according to the desired X-ray direction. r , sports.

[0056] Preferably, the robotic arm is a six-axis robotic arm.

[0057] Actual displacement of the ultrasonic probe The position of the ultrasound probe in the 6th joint coordinate system; the velocity of the ultrasound probe. In the formula,

[0058]

[0059] Preferably, in the second step, a subset of feature points are selected from all feature points through the following process: calculating the difference between the grayscale value of the pixel containing each feature point and the grayscale values ​​of the pixels above, below, left, and right to obtain the grayscale gradient value G. n Calculate the average gradient value of all feature points. Feature points with gradient values ​​greater than the average gradient value are retained. When a feature point in a region is retained multiple times, the multiple feature points in that region are merged. A circle of a certain diameter is used as the merging region, and the feature point closest to the center of the circle is selected and retained, finally obtaining the feature point TD.

[0060] The present invention also provides a method for detecting weld defects based on robot-mounted vision, which applies any of the above-mentioned methods, to detect weld defects in reactor pressure vessels.

[0061] The beneficial effects of this invention are that it replaces conventional manual inspection methods, using robots to automatically and intelligently inspect weld quality. This results in high inspection efficiency, high accuracy, and improved safety.

[0062] The macro-micro combined traversal method uses the robot's path planning as the macro-traversal and the robotic arm's trajectory planning as the micro-traversal, ensuring the integrity of weld inspection.

[0063] By incorporating a pressure sensor and combining it with impedance control, a force-position hybrid control system was proposed, with position control as the inner loop and force control as the outer loop. This system ensures that the ultrasonic testing probe can adhere to the wall surface with stable pressure, thereby improving the testing quality, stability, and reliability.

[0064] Further features and aspects of the present invention will be clearly described in the following detailed description with reference to the accompanying drawings. Attached Figure Description

[0065] Figure 1 Figure (1) shows the path of the robot traversing the welds on the reactor pressure vessel. In Figure (1), the red line represents the weld and the blue line represents the order of the various locations. Figure (2) shows a schematic diagram of the four horizontal welds on the upper part of the reactor pressure vessel. Figure (3) shows the order of the four horizontal welds in the region.

[0066] Figure 2 This is a schematic diagram of stamping 12 numbers at certain intervals along a weld seam path;

[0067] Figure 3 This is a schematic diagram showing the digital stamp located on the center line of the weld.

[0068] Figure 4 This is a flowchart of the weld inspection method;

[0069] Figure 5 This is the architecture diagram of the DBnet model;

[0070] Figure 6 This is the result of the stamped digital inspection;

[0071] Figure 7 This is a structural diagram of the robot;

[0072] Figure 8 This is a schematic diagram showing the state of the ultrasonic probe attached to the weld surface.

[0073] Figure 9 This is a schematic diagram of the ultrasonic probe detection principle;

[0074] Figure 10 This is a schematic diagram of force-position control based on impedance control;

[0075] Figure 11 Figure (a) shows the feature points generated by the stamped digits in the world coordinate system, where Figure (b) shows the extracted feature points TD and Figure (a) shows the extracted feature points Td.

[0076] Figure 12 It is a schematic diagram of the trajectory formed by the center of the stamped digits.

[0077] Explanation of symbols in the diagram:

[0078] 1-1. Circular pipe weld with a diameter of 1.31m; 1-2. Circular pipe weld with a diameter of 1.31m; 1-3. Circular pipe weld with a diameter of 0.53m; 1-4. Circular weld with a diameter of 4.3m; 1-5. Circular weld with a diameter of 4.3m; 1-6. Circular pipe weld with a diameter of 0.53m; 2. Weld surface; 3. Defects; 4. Trajectory formed by the center of the stamped numerals.

[0079] 10. Weld seam inspection robot; 10-1. Wall-climbing robot; 10-2. Global binocular camera; 10-3. Six-axis robotic arm; 10-4. Arm-mounted camera; 10-5. Ultrasonic probe; 10-6. Six-dimensional force sensor. Detailed Implementation

[0080] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0081] The inner wall of the reactor pressure vessel has multiple welds, so each weld needs to be inspected for defects individually. Currently, the welds are specially treated so that their surfaces are smooth and the welding marks are not visible, making it impossible to directly identify the welds using machine vision.

[0082] like Figure 7 As shown, the weld inspection robot 10 includes a wall-climbing robot 10-1, a global binocular camera 10-2, a six-axis robotic arm 10-3, an arm-mounted camera 10-4, an ultrasonic probe 10-5, and a six-dimensional force sensor 10-6. The global binocular camera 10-2 is connected to the wall-climbing robot 10-1, the six-axis robotic arm 10-3 is connected to the wall-climbing robot 10-1, the arm-mounted camera 10-4 is connected to the end effector of the six-axis robotic arm 10-3, the six-dimensional force sensor 10-6 is connected to the end effector of the six-axis robotic arm 10-3, and the ultrasonic probe 10-5 is connected to the six-dimensional force sensor 10-6. The wall-climbing robot 10-1 moves along the inner wall of the reactor pressure vessel, thereby moving the entire weld inspection robot 10. The arm-mounted camera 10-4 is used to acquire images of the actual working status of the ultrasonic probe at the end effector of the robotic arm for visual observation by the operator.

[0083] The main process of weld defect detection methods is as follows:

[0084] The first step is to determine the inspection area and plan a predetermined path based on the location of each weld seam trajectory.

[0085] refer to Figure 1In Figure (1), the red lines represent welds. The upper part of the reactor pressure vessel has three welds: 1-1 (circular nozzle weld with a diameter of 1.31m), 1-2 (circular nozzle weld with a diameter of 1.31m), 1-3 (circular nozzle weld with a diameter of 0.53m), and 1-6 (circular nozzle weld with a diameter of 0.53m). The lower part of the reactor pressure vessel has two welds: 1-4 (circular weld with a diameter of 4.3m) and 1-5 (circular weld with a diameter of 4.3m).

[0086] Therefore, the circular pipe weld 1-1 with a diameter of 1.31m is designated as the first location area, the circular pipe weld 1-3 with a diameter of 0.53m is designated as the second location area, the circular pipe weld 1-2 with a diameter of 1.31m is designated as the third location area, the circular pipe weld 1-6 with a diameter of 0.53m is designated as the fourth location area, the circular pipe weld 1-4 with a diameter of 4.3m is designated as the fifth location area, and the circular pipe weld 1-5 with a diameter of 4.3m is designated as the sixth location area, thus forming a predetermined path containing six location areas. Figure 1 The blue lines indicate the order of the various position areas: first the first position area, then the second, then the third, then the fourth, then the fifth, and finally the sixth.

[0087] The second step involves controlling the robot to move along a pre-planned path. The robot moves to the first location area (i.e., the circular pipe weld 1-1 with a diameter of 1.31m) to detect weld defects. After the detection is completed, the robot moves to the second location area (the circular pipe weld 1-3 with a diameter of 0.53m) to detect weld defects, then to the third location area (the circular pipe weld 1-2 with a diameter of 1.31m) to detect weld defects, then to the fourth location area (the circular pipe weld 1-6 with a diameter of 0.53m) to detect weld defects, then to the fifth location area (the circular weld 1-4 with a diameter of 4.3m) to detect weld defects, and finally to the sixth location area (the circular weld 1-5 with a diameter of 4.3m) to detect weld defects.

[0088] The third step involves controlling the movement of the six-axis robotic arm in each location area, with the ultrasonic probe at the end of the six-axis robotic arm detecting defects in the weld.

[0089] Considering the special treatment performed to smooth the surface of the weld, this invention uses a digital stamping method to locate the weld. A digital stamp is applied at regular intervals along the weld trajectory, forming several consecutive digital stamps, such as... Figure 2 As shown ( Figure 2 The display shows 12 stamped numbers, from 1 to 12. (Reference) Figure 3The digital stamp is located on the center line of the weld. Therefore, the trajectory formed by the digital stamp represents the weld trajectory.

[0090] Beforehand, photograph all the digital stamps using a camera to obtain a two-dimensional image of each stamp. For each two-dimensional image, the SIFT algorithm is used to extract all feature points of the stamp's digits. Furthermore, the grayscale gradient value G is obtained by calculating the difference between the grayscale value of the pixel containing each feature point and the grayscale values ​​of the pixels on its surrounding edges. n Calculate the average gradient value of all feature points. Feature points with gradient values ​​greater than the average gradient value are retained as those showing significant grayscale changes. When a feature point in a region is retained multiple times, it indicates that a significant feature exists in that region that can be frequently extracted. Therefore, multiple feature points in that region can be merged to represent this significant feature. A circle with a diameter of 30 pixels is used as the merging region. When there are more than two feature points in this region, the feature point closest to the center of the circle is selected and retained. Finally, feature points TD with significant grayscale changes and easy extraction are obtained, such as... Figure 11 Figure (a) shows the calculation of the coordinates of the selected feature points TD.

[0091] When the weld inspection robot 10 moves to the first location area (i.e., the circular nozzle weld 1-1 with a diameter of 1.31m), the global binocular camera 10-2 acquires images of the inner wall of the reactor pressure vessel, and the acquired images are sent to the controller. The controller uses the DBnet model and the CRNN model to identify the stamped numbers. DBnet is a segmentation-based text detection algorithm, referencing... Figure 5 This algorithm introduces a differentiable binarization module into the segmentation model, enabling the model to perform binarization using an adaptive threshold map. When performing recognition, the CRNN module first uses multiple convolutional layers to preview local features in the image, then employs an RNN layer to process sequential data and capture contextual information, and finally uses a fully connected layer to map the RNN layer output to character categories. Images acquired by a global stereo camera (10⁻²) are input to the DBnet model. The DBnet model outputs the pixel positions of the detected targets in the image and represents them with detection boxes. Then, the images within the detection boxes output by the DBnet model are input to the CRNN module. The CRNN module outputs the detection results of the stamped numbers, recognizing the stamped numbers, such as... Figure 6 As shown.

[0092] The position information of the stamped numbers is calculated based on the ranging principle of a binocular camera.

[0093] Establish a world coordinate system with the center of the stamped numerals as the origin.

[0094] The SIFT algorithm is used to extract all feature points Td of the stamped digits. Then, a brute-force matching method is used to match feature points Td with feature points TD, resulting in matching feature points Td for each feature point TD. Successfully matched feature points Td are retained. Feature points TD are those with significant grayscale changes and easy extraction after the initial screening; therefore, after matching, only feature points with significant grayscale changes and easy extraction are retained as feature points Td. Figure 11 As shown in Figure (b), these feature points Td are located in the world coordinate system. Calculate the coordinates of these feature points Td.

[0095] A matching point pair is a pair of feature points Td for each feature point TD. This matching point pair is the input for the subsequent PnP algorithm.

[0096] The matching point pairs are input into the PnP algorithm, which uses the PnP algorithm to solve the mapping relationship between feature point TD and feature point Td. That is, based on the known coordinates of feature point TD and feature point Td, the rotation matrix and translation matrix are calculated to obtain the mapping relationship.

[0097] The PnP algorithm is used to solve the above mapping relationship, and then the transformation matrix from the world coordinate system to the robot arm base coordinate system is obtained. A is the coordinate system of the robot arm base, and B is the world coordinate system.

[0098] In the aforementioned world coordinate system, the trajectory formed by the centers of all stamped numbers represents the trajectory of the weld seam, and trajectory information is established with the centers of all stamped numbers as the basis. like Figure 12 As shown. This trajectory information meets the following conditions:

[0099]

[0100] Where R is the radius of the trajectory, and h is the height of the trajectory in the pressure vessel.

[0101] By transforming the matrix Calculate trajectory information Trajectory information in the robot arm base coordinate system:

[0102] In trajectory information The resulting trajectory is sampled at certain intervals (e.g., 0.005m) to obtain multiple coordinate points, and these multiple coordinate points form the trajectory information. Track information Inverse kinematics yields the rotation angles of each joint of the six-axis robotic arm, and thus the motion trajectory of the end effector. This can be achieved through fifth-order polynomial programming, where the fifth-order polynomial is:

[0103]

[0104] To obtain a0, a1, a2, a3, a4, a5, starting from point t0 and target point t... f Construct equations to meet the constraints:

[0105]

[0106] By solving the equation, we can obtain a0, a1, a2, a3, a4, a5.

[0107] Next, the six-axis robotic arm is controlled to move, and the ultrasonic probe 10-5 at the end of the robotic arm moves according to the motion trajectory to detect defects in the weld. The state of the weld inspection by the ultrasonic probe 10-5 is as follows: Figure 8 As shown.

[0108] Ultrasonic testing is a non-destructive testing technique that uses high-frequency sound waves to detect internal defects in materials. (Reference) Figure 9 The working principle is that the piezoelectric crystal in the ultrasonic probe emits an ultrasonic pulse under the excitation of an electrical signal. The ultrasonic wave propagates in the weld material and encounters different structures or defects within the material (such as pores, cracks, lack of fusion, etc.). Due to the discontinuity of the material, the ultrasonic wave is reflected and scattered when it encounters defects. The ultrasonic probe also acts as a receiver, receiving the ultrasonic signals reflected back from within the material. The received electrical signal is amplified, filtered, and digitized, converting it into a signal spectrum that is easy to analyze. When the object under test does not contain defects, a strong bottom reflection echo can be received, with no other echoes between the initial reflected wave and the bottom reflection wave, such as... Figure 8 As shown in Figure (a). When the object under test has a defect, the difference between the acoustic impedance of the defect and the acoustic impedance of the material will cause sound wave reflection. When the defect is smaller than the beam width, a defect wave appears between the initial reflected wave and the bottom reflected wave, such as... Figure 9 As shown in Figure (b), the magnitude of the defect echo amplitude depends on the size of the projected area of ​​the defect in the direction of the incident sound beam. When a defect echo appears, the amplitude of the bottom echo will decrease accordingly. When the defect in the component is larger than the width of the sound beam, no incident wave reaches the bottom surface. At this time, there are only the initial wave and the defect wave, and no bottom echo will appear. Figure 8 As shown in Figure (c), the location and size of the weld are determined by the reflected echo from defects on the bottom surface or inside the inspected object, thus completing the ultrasonic detection of the weld interior.

[0109] In ultrasonic testing, ultrasonic waves experience minimal transmission loss in solids but attenuate rapidly in air. Therefore, the quality of the test largely depends on the adhesion between the ultrasonic probe and the surface of the object being inspected. To improve testing performance, a six-dimensional force sensor (10⁻⁶) is used. To control the ultrasonic probe's contact with the weld surface with stable pressure, an impedance control algorithm is employed, adding a force control loop to the robotic arm's position control loop. The impedance control algorithm analyzes the relationship between the current environment and the robot, indirectly controlling the contact force by comprehensively considering contact force control and the robot's end effector position control, rather than directly controlling the contact force between the robot and the environment. Impedance control makes the controlled object exhibit the mechanical characteristics of a second-order system consisting of mass M, damper B, and spring K. This system correlates the contact force information between the robot's end effector and the external environment with the end effector's displacement information, and this correlation can be changed by altering its parameters. In actual robot operation environments, force is typically used as the robot's input, and displacement as its output, exhibiting admittance characteristics, which is position-based impedance control. (Reference) Figure 10 The purpose of position-based impedance control is to establish the desired dynamic relationship between the actual position of the robot's end effector and the contact force, thereby achieving adjustment or tracking control of the contact force. The core formula for impedance control is:

[0110]

[0111] Where M, B, and K represent the mass parameter, damping parameter, and stiffness parameter, respectively. The mass parameter M is the mass of the six-axis robotic arm 10-3 itself, the damping parameter B is a set value, and the stiffness parameter K is a set value; X, It is the actual displacement, velocity, and acceleration of the ultrasonic probe, X. r , It represents the desired displacement, velocity, and acceleration of the ultrasonic probe; F is the actual pressure, i.e., the magnitude of the force fed back by the six-dimensional force sensor (10⁻⁶). r The desired pressure is a set value, such as around 5N.

[0112] Actual displacement X and velocity of the ultrasonic probe acceleration The angular velocity of a six-axis robotic arm is calculated by measuring the angles, angular velocities, and angular accelerations of each joint. The joint's photoelectric encoder detects the joint's angle θ, and the angular velocity is obtained by calculating the difference between angle θ and the calculated angle θ. angular velocity The angular acceleration is obtained by finite difference. The actual displacement X of the ultrasonic probe is obtained based on the forward kinematics of the robotic arm. The robotic arm in this embodiment has 6 joints, n=6, so This represents the position of the ultrasound probe in the 6th joint coordinate system. Let be the transformation matrix from joint i-1 to joint i.

[0113]

[0114] Where, α i-1 ,θ i ,d i ,a i-1 The DH parameters for the robotic arm can be obtained through the DH table.

[0115] Ultrasonic probe speed Obtained through the Jacobian matrix of the robotic arm.

[0116] in,

[0117]

[0118] In K(θ), each term represents the partial derivative of the ultrasound probe position with respect to the corresponding joint angle. In this embodiment, the robotic arm has 6 joints, therefore:

[0119]

[0120] speed Velocity has three directions: x, y, and z, so velocity... It is actually a 3D vector

[0121] acceleration of the ultrasonic probe By speed Taking the derivative, we get

[0122] Desired acceleration of the ultrasonic probe It is calculated using the following formula:

[0123]

[0124] Desired speed of ultrasonic probe It is about acceleration The result is obtained by integration, specifically by calculation using the following formula:

[0125]

[0126] Desired displacement X of the ultrasonic probe r Regarding the expected speed The result is obtained by integration, specifically by calculation using the following formula:

[0127]

[0128] X r , The angles, velocities, and accelerations of each joint of the robotic arm are calculated using the Jacobian matrix based on inverse kinematics. These angles, velocities, and accelerations are then input to the motion controller, which controls the robotic arm's movements to ensure the ultrasound probe moves according to the desired X-ray direction. r , Movement. X r , The process of calculating using the Jacobian matrix in inverse kinematics is as follows: Given the desired displacement of the robotic arm's end effector, solve for the joint angles using inverse kinematics. Then, using a numerical method, obtain the inverse solution: q t+1 =q t -J(θ) -1 F(θ), where F(θ) is the difference between the current pose of the robotic arm and the target pose, is solved through multiple iterations. t+1 The joint angle is obtained upon convergence. The joint angular velocity can be calculated from the desired velocity of the robotic arm's end effector. Similarly, the acceleration equation can be solved. Joint angular acceleration can be obtained.

[0129] When inspecting weld defects in the fifth location area (circular weld 1-4 with a diameter of 4.3m), the movement range of the six-axis robotic arm 10-3 could not cover the entire 4.3m diameter circular weld 1-4 due to its large size. Therefore, multiple stamped numbers were identified, and the centers of these numbers formed an arc-shaped trajectory. The ultrasonic probe moved along this trajectory. After inspecting the first arc-shaped trajectory, the second arc-shaped trajectory was identified, and the ultrasonic probe moved along it again. This process was repeated, forming the third and fourth arc-shaped trajectories, until the entire 4.3m diameter circular weld 1-4 was inspected. It is evident that the entire 4.3m diameter circular weld 1-4 was divided into multiple arc-shaped segments for inspection.

[0130] Similarly, the inspection of the sixth location area (circular weld 1-5 with a diameter of 4.3m) was also carried out in segments.

[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and variations.

Claims

1. A method for detecting weld defects based on a robot equipped with vision, characterized in that, The weld inspection robot includes a wall-climbing robot, a global binocular camera, a robotic arm, and an ultrasonic probe. The global binocular camera is connected to the wall-climbing robot, the robotic arm is connected to the wall-climbing robot, and the ultrasonic probe is connected to the end of the robotic arm. The weld defect detection method Includes the following steps: The first step is to stamp several consecutive numbers along the weld seam at certain intervals, with the numbers located on the center line of the weld seam. The second step is to take pictures of all the digital stamps with a camera beforehand to obtain a two-dimensional image of each digital stamp. For each two-dimensional image of the digital stamp, the SIFT algorithm is used to extract all the feature points of the stamp number. From all the feature points, some feature points are selected as feature points TD. The coordinates of the feature points TD are calculated. The third step involves the wall-climbing robot moving the weld inspection robot to several consecutive digit stamps. The global binocular camera captures images of the area to be inspected, and the captured images are sent to the controller. The controller identifies the stamp numbers in the images, calculates the position information of the stamp numbers, and establishes a world coordinate system with the center of the stamp numbers as the origin. The fourth step involves extracting all feature points Td of the stamped digits obtained in the third step using the SIFT algorithm. Then, the feature points Td and TD are matched using a brute-force matching method to obtain the feature point Td corresponding to each feature point TD as a matching point pair, and the successfully matched feature points Td are retained. The coordinates of the feature point Td in the world coordinate system are calculated. The fifth step is to use the PnP algorithm to solve the mapping relationship between feature point TD and feature point Td based on the matching point pairs; The sixth step involves solving the mapping relationship using the PnP algorithm to obtain the transformation matrix from the world coordinate system to the robot arm base coordinate system. Step 7: Establish trajectory information in the world coordinate system, using the centers of all the stamped numbers obtained in step 3. The trajectory information meets the following conditions: Where R is the radius of the trajectory, and h is the height of the trajectory; By transforming the matrix Calculate trajectory information Trajectory information in the robot arm base coordinate system: In trajectory information The resulting trajectory is obtained by sampling multiple coordinate points at certain intervals, and these multiple coordinate points form trajectory information. Track information The rotation angles of each joint of the robotic arm are obtained by inverse kinematics, and then the motion trajectory of the robotic arm's end effector is obtained. The eighth step involves controlling the robotic arm's movements. The ultrasonic probe at the end of the robotic arm moves along the motion trajectory and performs defect detection.

2. The weld defect detection method based on robot-mounted vision according to claim 1, characterized in that, In the second step, a subset of feature points are selected from all feature points through the following process: The grayscale gradient value G is obtained by calculating the difference between the grayscale value of the pixel containing each feature point and the grayscale values ​​of the pixels above, below, to the left, and to the right. n Calculate the average gradient value of all feature points. Feature points with gradient values ​​greater than the average gradient value are retained. When a feature point in a region is retained multiple times, the multiple feature points in that region are merged. A circle of a certain diameter is used as the merging region, and the feature point closest to the center of the circle is selected and retained, finally obtaining the feature point TD.

3. The weld defect detection method based on robot-mounted vision according to claim 1, characterized in that, In the third step, the DBnet model and CRNN model are used to identify the stamped numbers. The image acquired by the global binocular camera is input to the DBnet model. The DBnet model outputs the pixel position of the detected target in the image and represents it with a detection box. Then, the image within the detection box output by the DBnet model is input to the CRNN module. The CRNN module outputs the detection result of the stamped numbers and identifies the stamped numbers.

4. The weld defect detection method based on robot-mounted vision according to claim 1, characterized in that, The weld trajectory is a circular weld trajectory.

5. The weld defect detection method based on robot-mounted vision according to claim 1, characterized in that, The robotic arm is equipped with an arm-mounted camera at its end.

6. The weld defect detection method based on robot-mounted vision according to claim 1, wherein the robotic arm is a six-axis robotic arm.

7. A method for detecting weld defects based on a robot equipped with vision, characterized in that, The weld inspection robot includes a wall-climbing robot, a global binocular camera, a robotic arm, an ultrasonic probe, and a six-dimensional force sensor. The global binocular camera is connected to the wall-climbing robot, the robotic arm is connected to the wall-climbing robot, the six-dimensional force sensor is connected to the end effector of the robotic arm, and the ultrasonic probe is connected to the six-dimensional force sensor. The weld defect detection method Includes the following steps: The first step is to stamp several consecutive numbers along the weld seam at certain intervals, with the numbers located on the center line of the weld seam. The second step is to take pictures of all the digital stamps with a camera beforehand to obtain a two-dimensional image of each digital stamp. For each two-dimensional image of the digital stamp, the SIFT algorithm is used to extract all the feature points of the stamp number. From all the feature points, some feature points are selected as feature points TD. The coordinates of the feature points TD are calculated. The third step involves the wall-climbing robot moving the weld inspection robot to several consecutive digit stamps. The global binocular camera captures images of the area to be inspected, and the captured images are sent to the controller. The controller identifies the stamp numbers in the images, calculates the position information of the stamp numbers, and establishes a world coordinate system with the center of the stamp numbers as the origin. The fourth step involves extracting all feature points Td of the stamped digits obtained in the third step using the SIFT algorithm. Then, the feature points Td and TD are matched using a brute-force matching method to obtain the feature point Td corresponding to each feature point TD as a matching point pair, and the successfully matched feature points Td are retained. The coordinates of the feature point Td in the world coordinate system are calculated. The fifth step is to use the PnP algorithm to solve the mapping relationship between feature point TD and feature point Td based on the matching point pairs; The sixth step involves solving the mapping relationship using the PnP algorithm to obtain the transformation matrix from the world coordinate system to the robot arm base coordinate system. A B T; Step 7: Establish trajectory information in the world coordinate system, using the centers of all the stamped numbers obtained in step 3. The trajectory information meets the following conditions: Where R is the radius of the trajectory, and h is the height of the trajectory; By transforming the matrix Calculate trajectory information Trajectory information in the robot arm base coordinate system: In trajectory information The resulting trajectory is obtained by sampling multiple coordinate points at certain intervals, and these multiple coordinate points form trajectory information. Track information The rotation angles of each joint of the robotic arm are obtained by inverse kinematics, and then the motion trajectory of the robotic arm's end effector is obtained. The eighth step involves using impedance control to control the robotic arm's movements. The ultrasonic probe at the end of the robotic arm moves along the motion trajectory and performs defect detection. The formula for impedance control is: Wherein, mass parameter M is the mass of the robotic arm itself, damping parameter B is a set value, and stiffness parameter K is a set value; These are the actual displacement, velocity, and acceleration of the ultrasonic probe. F represents the desired displacement, velocity, and acceleration of the ultrasonic probe; F is the actual pressure fed back by the six-dimensional force sensor; F r The expected pressure set; Actual displacement X and velocity of the ultrasonic probe acceleration The angular velocity of the robotic arm is calculated by measuring the angles, angular velocities, and angular accelerations of each joint. The joint's photoelectric encoder detects the joint's angle θ, and the angular velocity is obtained by calculating the difference between angle θ and the calculated angle θ. angular velocity The angular acceleration is obtained by finite difference. Calculate the actual displacement X of the ultrasonic probe. Let be the transformation matrix from joint i-1 to joint i. Where, α i-1 ,θ i ,d i ,a i-1 To establish the DH parameters for the robotic arm; Ultrasonic probe speed Obtained through the Jacobian matrix of the robotic arm. in, In J(θ), each term represents the partial derivative of the ultrasound probe position with respect to the corresponding joint angle. Acceleration of the ultrasonic probe By speed Taking the derivative, we get Desired acceleration of the ultrasonic probe It is calculated using the following formula: Desired speed of ultrasonic probe It is calculated using the following formula: Desired displacement X of the ultrasonic probe r It is calculated using the following formula: Will The angles, velocities, and accelerations of each joint of the robotic arm are calculated using the Jacobian matrix based on inverse kinematics. These angles, velocities, and accelerations are then input to the motion controller, which controls the robotic arm's movements to ensure the ultrasonic probe moves as desired. sports.

8. The weld defect detection method based on robot-mounted vision according to claim 7, characterized in that, The robotic arm is a six-axis robotic arm. Actual displacement of the ultrasonic probe The position of the ultrasound probe in the 6th joint coordinate system; the velocity of the ultrasound probe. In the formula, 9. The weld defect detection method based on robot-mounted vision according to claim 8, characterized in that, In the second step, a subset of feature points are selected from all feature points through the following process: The grayscale gradient value G is obtained by calculating the difference between the grayscale value of the pixel containing each feature point and the grayscale values ​​of the pixels above, below, to the left, and to the right. n Calculate the average gradient value of all feature points. Feature points with gradient values ​​greater than the average gradient value are retained. When a feature point in a region is retained multiple times, the multiple feature points in that region are merged. A circle of a certain diameter is used as the merging region, and the feature point closest to the center of the circle is selected and retained, finally obtaining the feature point TD.

10. A method for detecting weld defects in reactor pressure vessels, characterized in that, The weld defect detection method based on robot-mounted vision, as described in any one of claims 1-9, is used to inspect the welds of a reactor pressure vessel.

Images