Weld tracking method and system based on molten pool image, electronic device, electronic medium

Abstract

The application discloses a kind of based on molten pool image's weld tracking method and system, electronic equipment, storage medium, based on molten pool image's weld tracking method of the present application first real-time acquisition molten pool image and it is preprocessed, molten pool image is converted into gray image, then based on the molten pool image after pre-processing automatically identifies molten pool centroid position, then based on the molten pool image after pre-processing and molten pool centroid position automatically extracts weld feature straight line, finally according to the deviation distance of molten pool centroid position and weld feature straight line, welding position is corrected in real time, can be based on the real-time acquisition molten pool image welding position is corrected in real time, high-precision weld automatic tracking and deviation correction in welding process are realized.

Description

Technical Field

[0001] This invention relates to the field of welding technology, and in particular to a weld seam tracking method and system based on molten pool images, electronic equipment, and computer-readable storage medium. Background Technology

[0002] Automated welding technology has been widely applied in aerospace, construction, and machinery manufacturing, while weld seam tracking technology can automatically detect and correct the welding position during the welding process, ensuring that the welding robot always welds along the weld seam and thus guaranteeing welding quality. However, current weld seam tracking methods generally use deep learning models to identify the molten pool and weld seam based on acquired images to achieve weld seam tracking control. For example, patent CN113828892A discloses a molten pool center recognition system and weld seam tracking method based on HDR images. It uses pre-trained molten pool edge recognition models and weld seam edge recognition models to identify the molten pool edge and weld seam edge in the image to obtain the deviation between the molten pool center and the weld seam, and then corrects the welding torch position in real time based on the deviation to achieve weld seam tracking. However, existing deep learning model-based weld seam tracking methods require pre-prepared datasets to train the model, resulting in long processing times, significant influence from input samples, and poor weld seam tracking control accuracy in actual welding processes. Summary of the Invention

[0003] This invention provides a weld seam tracking method and system based on molten pool images, an electronic device, and a computer-readable storage medium. It can correct the welding position in real time based on real-time acquired molten pool images, realizing high-precision automatic tracking and correction of weld seams during the welding process.

[0004] According to one aspect of the present invention, a weld seam tracking method based on molten pool images is provided, comprising the following:

[0005] Real-time acquisition and preprocessing of molten pool images convert them into grayscale images;

[0006] The location of the centroid of the molten pool was identified based on the preprocessed molten pool image;

[0007] Weld feature lines are extracted based on the preprocessed molten pool image and the position of the molten pool centroid.

[0008] The welding position is corrected in real time based on the deviation distance between the centroid of the molten pool and the characteristic straight line of the weld.

[0009] Furthermore, the process of identifying the centroid location of the molten pool based on the preprocessed molten pool image includes the following:

[0010] The preprocessed molten pool image is binarized based on the percentage of pixels.

[0011] The area filter and shape filter are applied sequentially to the binarized molten pool image to extract the molten pool region.

[0012] The centroid position of the molten pool region is calculated by using the pixel values ​​of each location within the molten pool region as weighting coefficients.

[0013] Furthermore, the process of binarizing the preprocessed molten pool image based on the pixel count ratio includes the following:

[0014] Traverse the pixel values ​​of all pixels in the molten pool image, count the number of pixels for each pixel value, and select the pixel values ​​as the target segmentation threshold in descending order. Pixel values ​​of 255 are not included in the target segmentation threshold. Calculate the percentage of pixels with values ​​greater than the target segmentation threshold. If the percentage of pixels with a value greater than a preset percentage threshold is greater than the target segmentation threshold, then use the target segmentation threshold as the binarization segmentation threshold and perform binarization segmentation on the molten pool image.

[0015] Furthermore, the centroid position of the molten pool region is calculated based on the following formula:

[0016]

[0017] Among them, (x c y c (x) represents the coordinates of the molten pool centroid. i y i I(x) represents the position coordinates of the i-th pixel within the molten pool region. i y i ) represents the pixel value of the i-th pixel within the molten pool region.

[0018] Furthermore, the process of extracting the weld feature line based on the preprocessed molten pool image and the molten pool centroid position includes the following:

[0019] Perform Discrete Fourier Transform on the preprocessed molten pool image;

[0020] The molten pool image after discrete Fourier transform is first segmented into 3×3 regions, resulting in 9 segmented regions. The next detection region is determined based on the region where the centroid of the molten pool is located.

[0021] Each detection region is divided into 9 segments by a second 3×3 segmentation process. Variance detection and pixel peak detection are performed on each segmentation region to determine the next detection region.

[0022] Each detection region is divided into 9 segments by a third 3×3 segmentation process. Variance detection and pixel peak detection are then performed on each segmented region to determine the final detection region.

[0023] Spot detection and contour recognition are performed on the final detection area to extract the center position coordinates of the contour, and straight line fitting arrays for the left bevel and the right bevel of the weld are constructed respectively.

[0024] The left and right straight lines of the weld bevel are obtained by fitting the straight line fitting arrays of the left and right bevels of the weld bevel, respectively, and the characteristic straight lines of the weld bevel are determined based on the left and right straight lines of the weld bevel.

[0025] Furthermore, after the first 3×3 segmentation, if the centroid of the molten pool is located in the left three-grid area, only the right straight line of the weld bevel needs to be extracted; if the centroid of the molten pool is located in the right three-grid area, only the left straight line of the weld bevel needs to be extracted; if the centroid of the molten pool is located in the middle three-grid area, both the left and right straight lines of the weld bevel need to be extracted simultaneously.

[0026] Furthermore, the pixel peak detection process includes the following:

[0027] The process iterates through all pixels within the segmented region, recording the grayscale value of each pixel and counting the number of pixels with each grayscale value. For grayscale values ​​greater than 0 and less than 255, contrast is calculated. If the sum of the contrast values ​​is less than a preset contrast threshold, the segmented region is marked as a detection region; otherwise, it is excluded from the detection region. The formula for calculating contrast is:

[0028]

[0029] Among them, C i H(i) represents the contrast of grayscale value i, and H(i) represents the number of pixels with grayscale value i.

[0030] In addition, the present invention also provides a weld seam tracking system based on molten pool images, comprising:

[0031] The molten pool image acquisition module is used to acquire molten pool images in real time and preprocess them, converting the molten pool images into grayscale images;

[0032] The molten pool centroid location identification module is used to identify the location of the molten pool centroid based on the preprocessed molten pool image.

[0033] The weld feature line extraction module is used to extract weld feature lines based on the preprocessed molten pool image and the position of the molten pool centroid.

[0034] The real-time welding position correction module is used to correct the welding position in real time based on the deviation distance between the centroid of the molten pool and the characteristic straight line of the weld.

[0035] In addition, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the method described above by calling the computer program stored in the memory.

[0036] In addition, the present invention provides a computer-readable storage medium for storing a computer program for weld seam tracking based on a molten pool image, wherein the computer program executes the steps of the method described above when run on a computer.

[0037] The present invention has the following beneficial effects:

[0038] The weld seam tracking method based on molten pool images of the present invention first acquires molten pool images in real time and preprocesses them to convert the molten pool images into grayscale images. Then, the centroid position of the molten pool is automatically identified based on the preprocessed molten pool images. Next, the weld seam feature line is automatically extracted based on the preprocessed molten pool images and the centroid position of the molten pool. Finally, the welding position is corrected in real time according to the deviation distance between the centroid position of the molten pool and the weld seam feature line. The welding position can be corrected in real time based on the real-time acquired molten pool images, realizing high-precision automatic weld seam tracking and correction during the welding process.

[0039] In addition, the weld seam tracking system based on molten pool images of the present invention also has the above-mentioned advantages.

[0040] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0041] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0042] Figure 1 This is a schematic flowchart of a preferred embodiment of the weld seam tracking method based on molten pool images in this application;

[0043] Figure 2 This is a schematic diagram of a molten pool image acquired in a preferred embodiment of this application;

[0044] Figure 3 yes Figure 1 A schematic diagram of the sub-process of step S2;

[0045] Figure 4 yes Figure 1A schematic diagram of the sub-process of step S3;

[0046] Figure 5 This is a schematic diagram of the image obtained by performing a discrete Fourier transform on the molten pool image in a preferred embodiment of this application;

[0047] Figure 6 This is a schematic diagram of the first 3×3 segmentation of the image in a preferred embodiment of this application;

[0048] Figure 7 In the preferred embodiment of this application, when the centroid of the molten pool is located at... Figure 6 A schematic diagram of the next detection area determined in region 2;

[0049] Figure 8 This is a schematic diagram of the image after a second 3×3 segmentation process in a preferred embodiment of this application;

[0050] Figure 9 This is a schematic diagram of the next detection area determined after the second detection in the preferred embodiment of this application;

[0051] Figure 10 This is a schematic diagram of the image after a third 3×3 segmentation process in a preferred embodiment of this application;

[0052] Figure 11 This is a schematic diagram showing the deviation between the actual welding path and the set initial path during experimental verification in a preferred embodiment of this application.

[0053] Figure 12 This is a schematic diagram of the module structure of a weld seam tracking system based on molten pool images, according to a preferred embodiment of this application. Detailed Implementation

[0054] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0055] Reference Figure 1 A preferred embodiment of this application provides a weld seam tracking method based on molten pool images, including the following:

[0056] Step S1: Acquire molten pool images in real time and preprocess them to convert the molten pool images into grayscale images;

[0057] Step S2: Identify the centroid position of the molten pool based on the preprocessed molten pool image;

[0058] Step S3: Extract the weld feature line based on the preprocessed molten pool image and the position of the molten pool centroid;

[0059] Step S4: Correct the welding position in real time based on the deviation distance between the centroid of the molten pool and the characteristic straight line of the weld.

[0060] It is understood that the weld seam tracking method based on molten pool image in this embodiment first acquires molten pool image in real time and preprocesses it to convert the molten pool image into a grayscale image. Then, it automatically identifies the centroid position of the molten pool based on the preprocessed molten pool image. Next, it automatically extracts the weld seam feature line based on the preprocessed molten pool image and the centroid position of the molten pool. Finally, it corrects the welding position in real time based on the deviation distance between the centroid position of the molten pool and the weld seam feature line. It can correct the welding position in real time based on the real-time acquired molten pool image, realizing high-precision automatic weld seam tracking and correction during the welding process.

[0061] In step S1, a camera is used to acquire images of the molten pool in real time during the welding process. The acquired molten pool images are as follows: Figure 2 As shown, to ensure image quality and processing speed, the molten pool image needs to be preprocessed. Specifically, the acquired molten pool image is first scaled, converting it to a width of w pixels and a height of h pixels. For example, w is set to 640 and h to 480. Then, the image is converted to grayscale.

[0062] In addition, such as Figure 3 As shown, in step S2, the process of identifying the centroid position of the molten pool based on the preprocessed molten pool image includes the following:

[0063] Step S21: Binarize the preprocessed molten pool image based on the pixel count ratio;

[0064] Step S22: Perform area filtering and shape filtering on the binarized molten pool image in sequence to extract the molten pool region;

[0065] Step S23: Using the pixel values ​​of each location point within the molten pool region as weighting coefficients, calculate the centroid position of the molten pool region.

[0066] Specifically, the process first iterates through the pixel values ​​of all pixels in the molten pool image, counting the number of pixels for each pixel value. Pixel values ​​are then selected sequentially from highest to lowest as the target segmentation threshold, excluding the pixel value 255. The percentage of pixels with values ​​greater than the target segmentation threshold is calculated. If this percentage exceeds a preset threshold, the target segmentation threshold is used as the binarization threshold, and the molten pool image is binarized. For example, let N be the number of pixels with pixel value i. i (0≤i≤255), set a preset threshold T1 for the percentage of pixels with a value of 255, and first calculate the percentage P of pixels with a value of 255.255 The calculation formula is: P 255 =N 255 / (w×h), if P 255 If P > T1, then the image data is considered erroneous, and the image will not undergo further processing; it will be processed as an NG (Not Given) image. 255 If T1 is less than or equal to 254, then calculate the percentage P of pixels with a value greater than 254. 254 :P 254 =(N 255 +N 254 ) / (w×h), if P 254 If the value is greater than T1, then the pixel value 254 is used as the binarization segmentation threshold and the melt pool image is binarized. Otherwise, the above steps are repeated until P is reached. k >T1, where, The pixel value k is used as the binarization segmentation threshold, and the melt pool image is subjected to binarization segmentation.

[0067] It is understood that this invention takes into account factors such as spatter and workpiece reflection that easily occur during welding. Existing conventional adaptive binarization segmentation methods (such as the mean iteration method and the maximum inter-class difference method) are easily affected by these factors, resulting in poor recognition stability. Furthermore, since the positions of the camera and the workpiece are fixed, the area ratio of the molten pool region in the overall image fluctuates relatively little, but the pixel value fluctuates significantly. Therefore, it is impossible to set a segmentation threshold based on human experience to perform binarization segmentation processing on the molten pool image. The binarization segmentation threshold determination algorithm of this invention uses the proportion of pixels with high pixel values ​​as an indicator to determine the segmentation threshold, making full use of the characteristics of high pixel values ​​and small area ratio of the molten pool region. Compared with existing segmentation threshold determination algorithms, it has the combined advantages of relatively small computational load (i.e., fast computation speed) and high computational accuracy (i.e., high segmentation accuracy).

[0068] In addition, Gaussian filtering can be applied to the image before binarization segmentation to remove noise interference and improve image quality. Furthermore, open-region processing can be performed on the binarized image after binarization segmentation.

[0069] Then, area filtering is applied to the binarized image. Specifically, each contour in the image is examined. When a contour is closed, it is considered a valid contour, and its area is calculated. If the area of ​​a valid contour is less than a preset area threshold, the region enclosed by that contour is determined not to be a molten pool region, and the contour is deleted. Furthermore, it is checked whether each valid contour contains other valid contours. If so, the internal contours are deleted. Then, the number of valid contours is checked. If the number is greater than 2, no-go (NG) processing is performed. If the number is 0, it means that no welding state has been detected. It can be understood that this invention can quickly identify the molten pool by performing area filtering on the binarized image.

[0070] Next, shape filtering is performed on the binarized image. Specifically, the perimeter of the effective contour (i.e., the melt pool contour) is first calculated using the following formula: Where P represents the perimeter of the effective contour, (x i ,y i () represents the coordinates of the i-th point on the effective contour, and N represents the total number of points on the effective contour; then, the roundness of the effective contour is calculated using the following formula: Among them, C ir The radius represents the roundness of the effective contour, and Area represents the area of ​​the effective contour. Since the molten pool has the highest brightness during welding, but its brightness fluctuates (specifically related to the droplet transfer method, exhibiting periodic changes), while other reflective areas are less bright than the molten pool, approaching its brightness at its darkest point. Furthermore, the molten pool contour is generally close to a circle / ellipse, occasionally exhibiting irregularities, while reflective and other interfering areas are generally irregular. Therefore, there are three scenarios: 1. The area brightness is sufficient; in this case, even if the roundness is insufficient, it is considered a molten pool with no problem. 2. The brightness is relatively dim, but the roundness is good; it is also considered a molten pool. 3. If both brightness and roundness are insufficient, it is very likely not a molten pool. Therefore, this invention considers the influence of brightness and calculates the average brightness value B within the effective contour. a This is used as a correction factor to adjust the roundness. The adjusted roundness is: R b =B a ×C ir Then use it as the final shape filtering parameter, and then use R b With preset threshold S T Compare, if R b <S T If the weld pool is identified, the weld pool area is extracted, and the image acquired at this time is marked as having a normal welding state; otherwise, the image acquired at this time is marked as abnormal. It can be understood that this invention uses the average brightness of the effective contour as a correction coefficient to correct for roundness, taking into account the influence of brightness and improving the accuracy of weld pool identification.

[0071] Finally, after identifying the molten pool, the centroid position of the molten pool region is calculated using the pixel values ​​of each point within the molten pool region (i.e., the effective contour) as weighting coefficients. Specifically, the centroid position of the molten pool region is calculated based on the following formula:

[0072]

[0073] Among them, (x c y c (x) represents the coordinates of the molten pool centroid. i y i I(x) represents the position coordinates of the i-th pixel within the molten pool region.i y i ) represents the pixel value of the i-th pixel within the molten pool region.

[0074] It is understood that when calculating the centroid position of the molten pool, this invention takes into account that the closer the pixel is to the center of the molten pool, the larger (brighter) the pixel value of the pixel is, and conversely, the farther away from the center of the molten pool, the smaller (darker) the pixel value of the pixel is. Moreover, the effective contour extracted after area filtering and shape filtering is not necessarily regular. In reality, points that are far away need to have their weight influence on the detection of the centroid position of the molten pool reduced. Therefore, this invention adds the pixel value as a weight coefficient of the position point for calculating the centroid position of the molten pool, thereby improving the accuracy of the calculation of the centroid position of the molten pool.

[0075] In addition, such as Figure 4 As shown, in step S3, the process of extracting the weld feature line based on the preprocessed molten pool image and the molten pool centroid position includes the following:

[0076] Step S31: Perform Discrete Fourier Transform on the preprocessed molten pool image;

[0077] Step S32: Perform the first 3×3 segmentation on the molten pool image after discrete Fourier transform to obtain 9 segmented regions, and determine the next detection region based on the region where the centroid of the molten pool is located;

[0078] Step S33: Perform a second 3×3 segmentation process on each detection region, dividing each detection region into 9 segmentation regions, and perform variance detection and pixel peak detection on each segmentation region to determine the detection region for the next step;

[0079] Step S34: Perform a third 3×3 segmentation process on each detection region, dividing each detection region into 9 segmentation regions, and perform variance detection and pixel peak detection on each segmentation region again to determine the final detection region;

[0080] Step S35: Perform spot detection and contour recognition on the final detection area, extract the center position coordinates of the contour, and construct the straight line fitting arrays for the left bevel of the weld and the right bevel of the weld respectively.

[0081] Step S36: Fit the left and right straight lines of the weld bevel based on the straight line fitting arrays of the left and right bevels of the weld, respectively, and determine the characteristic straight line of the weld based on the straight lines of the left and right bevels of the weld.

[0082] Specifically, the preprocessed molten pool image has a size of w×h. Taking the top left corner of the molten pool image as the zero point, the right side of the image as the positive width direction, and the bottom side as the positive height direction, a discrete Fourier transform is first performed on the preprocessed molten pool image, resulting in the image shown below. Figure 5 As shown.

[0083] Then, the molten pool image after discrete Fourier transform is subjected to a first 3×3 segmentation process, resulting in 9 segmented regions. Each segmented region is labeled, with n ranging from 1 to 9. The 3×3 segmentation process is as follows: Figure 6 As shown, the next detection area is determined based on the position of the molten pool centroid in the height direction. The next detection area is the lower part of the segmented region where the molten pool centroid is located and the adjacent regions on the left and right. For example, when the molten pool centroid is located in segmented region 2, the next detection areas are 1, 3, 4, 5, 6, 7, 8, and 9; when the molten pool centroid is located in segmented region 5, the next detection areas are 4, 6, 7, 8, and 9, and so on. Figure 7 As shown; when the centroid of the molten pool is located in segmented region 8, the next detection region will be 7 or 9.

[0084] Next, a second detection is performed, dividing each detection region into 9 sub-regions using a second 3x3 segmentation process. Figure 8 As shown, variance detection and pixel peak detection are performed on each segmented region to determine the next detection region, such as... Figure 9 As shown. In addition, after the second segmentation, a label nm can be generated for each segmented region, where n is the region label after the first segmentation and m is the region label after the second segmentation, with m taking values ​​from 1 to 9.

[0085] The variance detection process involves calculating the variance of pixel values ​​within each segmented region using the following formula: Where N represents the number of pixels in the segmented region, I i Let μ represent the pixel value of the i-th pixel within the segmented region, and let μ represent the average pixel value of the segmented region. Let σ represent the variance. 2 Compared with the preset variance threshold, if the variance σ 2 If the variance exceeds a preset threshold, the segmented region is marked as the next detection region; otherwise, it is excluded from the detection region.

[0086] In addition, the pixel peak detection process includes the following:

[0087] The algorithm iterates through all pixels within the segmented region, recording the grayscale value of each pixel and counting the number of pixels with each grayscale value. For grayscale values ​​greater than 0 and less than 255, contrast is calculated. If the sum of contrast values ​​within the segmented region is less than a preset contrast threshold, the segmented region is marked as the next detection region; otherwise, it is excluded from the detection region. The formula for calculating contrast is:

[0088]

[0089] Among them, C i H(i) represents the contrast of grayscale value i, and H(i) represents the number of pixels with grayscale value i.

[0090] It is understood that this invention takes into account the significant pixel value variations in weld seam areas. Regions with a detection variance greater than a variance threshold are highly likely to be weld seam areas. Furthermore, considering the continuity of weld seams, the extracted pixel peak values ​​in corresponding regions will be relatively similar, resulting in a relatively small sum of contrast for the segmented regions. If the sum of contrast for a segmented region is less than a preset threshold, the segmented region is considered a weld seam area. If the sum of contrast for a segmented region is greater than the preset threshold, it indicates poor continuity, and the segmented region is not considered a weld seam area. Therefore, this invention improves the detection accuracy of weld seam areas by performing variance detection and pixel peak detection on each segmented region.

[0091] Then, a third detection is performed, dividing each detection region into 9 sub-regions using a 3x3 segmentation process. Figure 10 As shown, variance detection and pixel peak detection are performed again on each segmented region to determine the final detection region. In addition, after the third segmentation, a label nml can be generated for each segmented region, where n is the region label after the first segmentation, m is the region label after the second segmentation, and l is the region label after the third segmentation, with l taking a value from 1 to 9.

[0092] Next, spot detection and contour recognition are performed on the final detection area to extract the center coordinates of the contour. Straight line fitting arrays for the left and right bevels of the weld are then constructed. The straight line fitting array for the left bevel is the array located in the region n = 1, 2, 4, 5, 7, 8, denoted as P. cl The linear fitting array for the right bevel of the weld is the array located in the region n = 2, 3, 5, 6, 8, 9, denoted as P. crFurthermore, the specific spot detection and contour recognition processes are existing technologies and will not be elaborated here. For example, OTSU threshold segmentation is performed first, followed by erosion and dilation operations on the segmented image to filter out smaller spots. Then, CANNY edge detection is performed on the image. After detection, contours in the image are identified, and the center position of the identified contours is detected and saved to array P. c In the middle, then P c Divided into P cl and P cr .

[0093] Finally, the left and right straight lines of the weld bevel are fitted based on the fitting arrays of the left and right bevel lines, respectively, and the characteristic straight lines of the weld are determined based on these lines. For example, assuming the fitted line on the left is L... l =ax + by + c, calculate P cl The distance from each point in the curve to the fitted line is: Solving this equation with the objective of minimizing the sum of distances, methods include but are not limited to linear programming and quadratic programming algorithms, yields the parameters a, b, and c. This provides the equation for the fitted line on the left, thus determining the left side of the weld bevel. Similarly, the right side of the weld bevel can be fitted using the same principle. Therefore, the left and right sides of the weld bevel represent the extreme positions reached by the molten pool. The fitted left side of the weld bevel, L... l and the straight line L on the right side of the weld bevel r Then, by calculating the straight line L l and line L r By finding the center point and the average slope, the median L between the two points can be calculated. t , center line L t As the characteristic straight line of the weld, the centerline L t The direction is the theoretical direction of advancement for welding.

[0094] It is understood that this invention performs three 3×3 image segmentation processes on the molten pool image based on the centroid position of the molten pool. After the first segmentation, the detection area for the next step can be quickly determined based on the height position region of the molten pool centroid. After the second and third segmentation processes, variance detection and pixel peak detection are performed on each segmented region to accurately extract the weld area, thereby precisely determining the theoretical direction of welding and improving the accuracy of welding position correction during the welding process. Furthermore, setting the image segmentation to three times is optimal. Too many segmentations introduce more image detail noise, affecting the accuracy of weld area recognition; too few segmentations lead to insufficient recognition accuracy, which also affects the accuracy of weld area recognition.

[0095] Optionally, after the first 3×3 segmentation, if the centroid position of the molten pool is x c Located in the left three squares area (i.e. Figure 6 When x is in regions 1, 4, and 7, that is, x c If w / 3, then only the straight line on the right side of the weld bevel needs to be extracted. If the centroid of the molten pool is located in the right third grid area (i.e. Figure 6 When the region is in the range of 3, 6, or 9, that is, x c If the weld bevel is greater than 2w / 3, then only the straight line on the left side of the weld bevel needs to be extracted. If the centroid of the molten pool is located in the middle three-cell area (i.e., Figure 6 When the regions are 2, 5, and 8, i.e., w / 3≤x c If the value is ≤2w / 3, then both the left and right straight lines of the weld bevel need to be extracted simultaneously. When only the right or left straight line of the weld bevel is extracted, it is directly used as the weld feature line.

[0096] It is understood that after the first 3×3 division process, if the centroid position of the molten pool is x c When located in the left three-grid area, only the middle three-grid area and the right three-grid area need to be inspected for weld seam area. If the centroid position of the molten pool is x c When the weld area is located in the right three grid area, only the left three grid area and the middle three grid area need to be inspected, which helps to reduce the amount of calculation and improve efficiency.

[0097] Additionally, in step S4, within each welding torch oscillation cycle, the line L from the centroid of the molten pool to the weld characteristic line is calculated. t The distance is used as a reference to correct the real-time position of the weld, with the initial distance as a benchmark. Correction methods include, but are not limited to, PID control and fuzzy control, thereby achieving automatic weld tracking and correction during the welding process, which is beneficial to improving welding accuracy. Furthermore, this invention has been experimentally verified. After setting the arc starting point and initial path (i.e., the weld characteristic straight line) on the test plate, automatic welding was performed on the test plate. During the welding process, the weld could be stably tracked, and the deviation between the actual welding path and the set initial path was small. Figure 11 As shown,

[0098] In addition, such as Figure 12 As shown, another embodiment of the present invention also provides a weld seam tracking system based on molten pool images, preferably employing the weld seam tracking method based on molten pool images as described above, including:

[0099] The molten pool image acquisition module is used to acquire molten pool images in real time and preprocess them, converting the molten pool images into grayscale images;

[0100] The molten pool centroid location identification module is used to identify the location of the molten pool centroid based on the preprocessed molten pool image.

[0101] The weld feature line extraction module is used to extract weld feature lines based on the preprocessed molten pool image and the position of the molten pool centroid.

[0102] The real-time welding position correction module is used to correct the welding position in real time based on the deviation distance between the centroid of the molten pool and the characteristic straight line of the weld.

[0103] As can be understood, the weld tracking system based on molten pool images in this embodiment first acquires molten pool images in real time and preprocesses them to convert the molten pool images into grayscale images. Then, it automatically identifies the centroid position of the molten pool based on the preprocessed molten pool images. Next, it automatically extracts the weld feature line based on the preprocessed molten pool images and the centroid position of the molten pool. Finally, it corrects the welding position in real time based on the deviation distance between the centroid position of the molten pool and the weld feature line. It can correct the welding position in real time based on the real-time acquired molten pool images, realizing high-precision automatic weld tracking and correction during the welding process.

[0104] In addition, another embodiment of the present invention provides an electronic device including a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the method described above by calling the computer program stored in the memory.

[0105] In addition, another embodiment of the present invention provides a computer-readable storage medium for storing a computer program for weld seam tracking based on a molten pool image, wherein the computer program executes the steps of the method described above when run on a computer.

[0106] Common computer-readable storage media include: floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tape, any other physical media with perforated patterns, random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), flash erasable programmable read-only memory (FLASH-EPROM), any other memory chips or cartridges, or any other media readable by a computer. Instructions may further be transmitted or received by a transmission medium. The term transmission medium can include any tangible or intangible medium used to store, encode, or carry instructions for machine execution, and includes digital or analog communication signals or intangible media that facilitate communication of such instructions. Transmission media include coaxial cables, copper wires, and optical fibers, which contain conductors for transmitting a bus of computer data signals.

[0107] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0108] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0109] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0111] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0112] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A weld seam tracking method based on molten pool images, characterized in that, Includes the following: Real-time acquisition and preprocessing of molten pool images convert them into grayscale images; The location of the centroid of the molten pool was identified based on the preprocessed molten pool image; Weld feature lines are extracted based on the preprocessed molten pool image and the position of the molten pool centroid. The welding position is corrected in real time based on the deviation distance between the centroid of the molten pool and the characteristic straight line of the weld. The process of extracting the weld feature line based on the preprocessed molten pool image and the centroid position of the molten pool includes the following: Perform Discrete Fourier Transform on the preprocessed molten pool image; The molten pool image after discrete Fourier transform is first segmented into 3×3 regions, resulting in 9 segmented regions. The next detection region is determined based on the region where the centroid of the molten pool is located. Each detection region is divided into 9 segments by a second 3×3 segmentation process. Variance detection and pixel peak detection are performed on each segmentation region to determine the next detection region. Each detection region is divided into 9 segments by a third 3×3 segmentation process. Variance detection and pixel peak detection are then performed on each segmented region to determine the final detection region. Spot detection and contour recognition are performed on the final detection area to extract the center position coordinates of the contour, and straight line fitting arrays for the left bevel and the right bevel of the weld are constructed respectively. The left and right straight lines of the weld bevel are obtained by fitting the straight line fitting arrays of the left and right bevels of the weld bevel, respectively, and the characteristic straight lines of the weld bevel are determined based on the left and right straight lines of the weld bevel.

2. The weld seam tracking method based on molten pool images as described in claim 1, characterized in that, The process of identifying the centroid location of the molten pool based on the preprocessed molten pool image includes the following: The preprocessed molten pool image is binarized based on the percentage of pixels. The area filter and shape filter are applied sequentially to the binarized molten pool image to extract the molten pool region. The centroid position of the molten pool region is calculated by using the pixel values ​​of each location within the molten pool region as weighting coefficients.

3. The weld seam tracking method based on molten pool images as described in claim 2, characterized in that, The process of binarizing the preprocessed molten pool image based on the pixel count ratio includes the following: Traverse the pixel values ​​of all pixels in the molten pool image, count the number of pixels for each pixel value, and select the pixel values ​​as the target segmentation threshold in descending order. Pixel values ​​of 255 are not included in the target segmentation threshold. Calculate the percentage of pixels with values ​​greater than the target segmentation threshold. If the percentage of pixels with a value greater than a preset percentage threshold is greater than the target segmentation threshold, then use the target segmentation threshold as the binarization segmentation threshold and perform binarization segmentation on the molten pool image.

4. The weld seam tracking method based on molten pool images as described in claim 2, characterized in that, The centroid location of the molten pool region is calculated based on the following formula: ； in,( x c , y c ) represents the coordinates of the centroid of the molten pool, ( x i , y i () represents the position coordinates of the i-th pixel within the molten pool region. I ( x i , y i ) represents the pixel value of the i-th pixel within the molten pool region.

5. The weld seam tracking method based on molten pool images as described in claim 1, characterized in that, After the first 3×3 segmentation, if the centroid of the molten pool is located in the left three-grid area, only the right straight line of the weld bevel needs to be extracted. If the centroid of the molten pool is located in the right three-grid area, only the left straight line of the weld bevel needs to be extracted. If the centroid of the molten pool is located in the middle three-grid area, both the left and right straight lines of the weld bevel need to be extracted.

6. The weld seam tracking method based on molten pool images as described in claim 1, characterized in that, The pixel peak detection process includes the following: The process iterates through all pixels within the segmented region, recording the grayscale value of each pixel and counting the number of pixels with each grayscale value. For grayscale values ​​greater than 0 and less than 255, contrast is calculated. If the sum of the contrast values ​​is less than a preset contrast threshold, the segmented region is marked as a detection region; otherwise, it is excluded from the detection region. The formula for calculating contrast is: ； in, C i This represents the contrast of the grayscale value i. H (i) represents the number of pixels with grayscale value i.

7. A weld seam tracking system based on molten pool images, employing the weld seam tracking method based on molten pool images as described in any one of claims 1 to 6, characterized in that, include: The molten pool image acquisition module is used to acquire molten pool images in real time and preprocess them, converting the molten pool images into grayscale images; The molten pool centroid location identification module is used to identify the location of the molten pool centroid based on the preprocessed molten pool image. The weld feature line extraction module is used to extract weld feature lines based on the preprocessed molten pool image and the position of the molten pool centroid. The real-time welding position correction module is used to correct the welding position in real time based on the deviation distance between the centroid of the molten pool and the characteristic straight line of the weld.

8. An electronic device, characterized in that, The method includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the method as described in any one of claims 1 to 6 by calling the computer program stored in the memory.

9. A computer-readable storage medium for storing a computer program for weld seam tracking based on molten pool images, characterized in that, The computer program, when run on a computer, performs the steps of the method as described in any one of claims 1 to 6.

Images