A Weld Seam Tracking Method and System Based on Neighborhood Search and Graph Optimization

Abstract

The present invention belongs to the technical field of welding vision sensors. In order to solve the problem that the existing weld tracking methods are difficult to adapt to the weld trajectories with high dynamic changes, a weld tracking method and system based on neighborhood search and graph optimization are provided. The weld tracking method based on neighborhood search and graph optimization includes converting the laser stripe image of the weld groove into a grayscale image; traversing the grayscale image pixel by pixel in both directions column by column to extract two centerlines; respectively extracting the inflection points on both sides of the weld groove based on the two centerlines and using them as the estimated positions of the weld center points; taking the estimated positions of the weld center points of each frame of grayscale image as nodes, and connecting the nodes in sequence according to the welding time order to construct edges, thus obtaining a graph structure; taking the measured positions of the center points of each frame of grayscale image and the smoothness of the estimated positions of the center points between adjacent frames of grayscale images as constraint relations, determining the smooth trajectory of the weld center points through graph optimization, and realizing the high-precision real-time tracking of the weld center points in combination with a sliding window.

Description

Technical Field

[0001] The present invention belongs to the technical field of welding vision sensors, and particularly relates to a weld seam tracking method and system based on neighborhood search and graph optimization. Background Technique

[0002] The statements in this part only provide background technical information related to the present invention, and do not necessarily constitute prior art.

[0003] In recent years, with the rapid development of robot technology and computer vision technology, the application scope of welding robots in industrial production has become wider and wider, and the application scenarios cover many aspects such as a large number of standardized components and small batch customized parts. At present, the operation control of welding robots still mainly relies on teaching programming and offline programming. In order to improve welding accuracy and programming efficiency, installing a line laser sensor at the end of the robotic arm for weld seam scanning and tracking has become an important development trend in the field of welding robots. The introduction of the line laser vision sensor provides a new solution for the real-time tracking of welding paths.

[0004] However, in the prior art, the efficiency of extracting the center point by completely traversing all column pixels of the image is low, and due to interference such as arc light noise and workpiece surface reflection in the welding environment, the quality of the weld seam image collected by the sensor may decline, resulting in low reliability of inflection point detection and it is difficult to achieve a balance between real-time performance and accuracy in a dynamic scene. In addition, the existing Kalman filtering method has limited control over long-term cumulative errors and is difficult to adapt to high-dynamic weld seam trajectories. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a weld seam tracking method and system based on neighborhood search and graph optimization, which improves the robustness and efficiency of center point extraction through dynamic neighborhood search and cost function design, and realizes high-precision real-time tracking of the weld seam center point by combining sliding window optimization.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The first aspect of the present invention provides a weld seam tracking method based on neighborhood search and graph optimization.

[0008] In one or more embodiments, a weld seam tracking method based on neighborhood search and graph optimization is provided, including:

[0009] Obtain the laser stripe image of the weld groove and convert it into a grayscale image;

[0010] Traverse the grayscale image pixel by pixel in both directions column by column; when traversing the image column by column, based on the center point position extracted from the previous column, dynamically determine the search neighborhood of the current column, calculate the cost of the pixels in the search neighborhood, and select the pixel with the lowest cost as the center point of the current column to obtain two center lines;

[0011] Extract the inflection points on both sides of the weld groove based on the two center lines respectively, and use the midpoint of the inflection points on both sides as the estimated position of the weld center point;

[0012] Use the estimated position of the weld center point of each frame of grayscale image as a node, and connect the nodes in sequence according to the welding time sequence to construct edges, obtaining a graph structure;

[0013] Use the measured position of the center point of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frames of grayscale images as constraint relationships, and optimize the state values of the nodes by minimizing the constraint errors of all edges in the graph structure within the sliding window, finally determining the smooth trajectory of the weld center point for welding path planning and weld tracking.

[0014] As an implementation, during the process of traversing the image column by column, if the distance between the position of the pixel with the largest grayscale value on the leftmost / rightmost side of the current frame image and the center point position of the previous frame exceeds a set number of pixels, it is determined that the center point selection is incorrect. At this time, directly use the position of the previous frame as the center point position of the current frame.

[0015] As an implementation, during the process of traversing the image column by column, the expression for the cost of the pixels in the search neighborhood is:

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] where, is the weight parameter; is the expression for the cost of the pixels in the search neighborhood; is the distance cost; is the grayscale cost; is the dynamic noise adjustment cost; is the pixel; is the center point of the previous column; is the grayscale value of the pixel point; is the deviation of the pixel point from the position change direction of the previous frame.

[0021] As an implementation method, if during the traversal of the current frame grayscale image, the proportion of pixels with a set high grayscale value in the search neighborhood of a certain column exceeds the set threshold, it is determined that the welding arc light noise partially or completely obscures the center line of the laser stripe within the search neighborhood, and the center point pixel position of the corresponding column in the current frame grayscale image is directly selected according to the moving direction of the center point pixel of the corresponding column in the previous frame grayscale image.

[0022] As an implementation method, if the center line of the current frame grayscale image is obscured by noise, the center line shape of the previous frame image not obscured by noise is used to supplement the current frame image;

[0023] As an implementation method, if the grayscale values of all pixels in the search neighborhood are less than the set threshold, it is determined that the laser stripe is interrupted, and the next column of pixels is continued to be processed; if the interruption occurs for a set number of consecutive times, it is determined that the laser stripe is completely interrupted, the traversal is stopped, and the column coordinates of the center points of each subsequent column are set to -1.

[0024] As an implementation method, based on the two center lines, a dynamic fitting method is respectively used to extract the inflection points on both sides of the weld groove, and the process is as follows:

[0025] Starting from the leftmost / rightmost end of the center line, a set number of points are selected in the front, and a straight line is fitted using the least squares method as the initial fitting result;

[0026] When traversing the subsequent points, the distance from the current point to the fitted straight line is judged point by point; if the distance from the point to the straight line is less than the preset threshold, the point is added to the current set of fitted points; when the number of newly added points reaches the set update threshold, the straight line is refitted;

[0027] If a set number of consecutive points deviate from the fitted straight line, it is determined that the center line bends at the current position, the fitting is stopped, and the current position is recorded as the inflection point.

[0028] As an implementation method, optimize the cost function corresponding to the state value of the node It is:

[0029] ;

[0030] Among them, the first item is the error between the measured position and the estimated position of the center point of the grayscale image; the second item is the smoothness constraint of the estimated position of the center point between adjacent frame grayscale images; is the measured position of the center point of the th grayscale image, is the estimated position of the center point of the th grayscale image; is the estimated position of the center point of the th grayscale image; The weight for the smoothness constraint.

[0031] The second aspect of the present invention provides a weld seam tracking system based on neighborhood search and graph optimization.

[0032] In one or more embodiments, a weld seam tracking system based on neighborhood search and graph optimization includes:

[0033] An image acquisition module, which is used to acquire the laser stripe image of the weld seam groove and convert it into a grayscale image;

[0034] A centerline extraction module, which is used to traverse the grayscale image pixel by pixel in both directions column by column; when traversing the image column by column, based on the center point position extracted from the previous column, dynamically determine the search neighborhood of the current column, calculate the cost of the pixels in the search neighborhood, select the pixel with the lowest cost as the center point of the current column, and obtain two centerlines;

[0035] A center point estimation module, which is used to extract the inflection points on both sides of the weld seam groove based on the two centerlines respectively, and use the midpoint of the inflection points on both sides as the estimated position of the weld seam center point;

[0036] A graph structure construction module, which is used to use the estimated position of the weld seam center point of each frame of grayscale image as a node, and sequentially connect the nodes in the order of welding time to construct edges, obtaining a graph structure;

[0037] A graph structure optimization module, which is used to use the smoothness of the center point measurement position of each frame of grayscale image and the estimated position of the center point between adjacent frames of grayscale images as constraint relationships, and optimize the state values of the nodes by minimizing the constraint errors of all edges in the graph structure within a sliding window, and finally determine the smooth trajectory of the weld seam center point for welding path planning and weld seam tracking.

[0038] The third aspect of the present invention provides a computer-readable storage medium.

[0039] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the weld seam tracking method based on neighborhood search and graph optimization as described above.

[0040] The fourth aspect of the present invention provides an electronic device.

[0041] An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements the steps in the weld seam tracking method based on neighborhood search and graph optimization as described above.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] (1) For the laser stripe image converted into a grayscale image, the present invention sequentially extracts the center line through dynamic neighborhood search, determines the estimated position of the weld center point through inflection point extraction, combines the graph structure and the sliding window to optimize the graph structure, determines the smooth trajectory of the weld center point, realizes the high-precision tracking of various welds in a high-noise environment, takes into account real-time performance and global accuracy, and finally improves the tracking accuracy and stability of the welding robot system.

[0044] (2) The present invention is no longer limited to identifying and extracting a single weld type, can accurately extract the weld center points of various typical welds (such as V-shaped, lap joint, fillet joint), realizes the dynamic detection and accurate extraction of inflection points, and effectively deals with sudden changes in weld shapes.

[0045] (3) In order to solve the situation that higher grayscale values may cover the grayscale values of laser stripes, resulting in incorrect selection of the center point, the present invention adds an error correction mechanism based on the position of the leftmost center point of the previous frame image, improving the accuracy of determining the weld center point.

[0046] (4) The present invention introduces a dynamic fitting mechanism, extracts the inflection points on both sides of the weld groove through the dynamic fitting method, improves the efficiency and accuracy of inflection point detection, enhances the fitting accuracy by reducing redundant calculations on long straight line segments, and enhances the robustness to complex weld shapes.

[0047] (5) The present invention comprehensively combines the distance cost, grayscale cost and dynamic noise adjustment cost to construct a cost function for pixels in the search neighborhood, considers the three aspects of distance, grayscale and dynamic noise, and realizes the high-precision tracking of various welds in a high-noise environment. Description of the Drawings

[0048] The specification drawings forming a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0049] Figure 1 is a schematic flowchart of the weld tracking method based on neighborhood search and graph optimization according to an embodiment of the present invention;

[0050] Figure 2 is a specific process diagram of the weld tracking based on neighborhood search and graph optimization according to an embodiment of the present invention;

[0051] Figure 3 is a schematic structural diagram of the weld tracking system based on neighborhood search and graph optimization according to an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of identifying inflection points for different groove welds in an embodiment of the present invention;

[0053] Figure 5It is a schematic diagram of intercepting a single weld groove image of the ROI in the embodiment of the present invention;

[0054] Figure 6 It is a schematic diagram that there is arc light noise in the first column on the left side of the weld image in the embodiment of the present invention;

[0055] Figure 7 It is a schematic diagram of laser stripe interruption in the embodiment of the present invention. Detailed implementation manners

[0056] The present invention will be further described below in conjunction with the drawings and embodiments.

[0057] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0058] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0059] Figure 1 It is a flow schematic diagram of a weld tracking method based on neighborhood search and graph optimization in the embodiment of the present invention. As Figure 1 shown, the weld tracking method based on neighborhood search and graph optimization in this embodiment may include:

[0060] S101, obtaining a laser stripe image of the weld groove and converting it into a grayscale image;

[0061] S102, traversing the grayscale image pixel by pixel in both directions by column; when traversing the image by column, based on the center point position extracted from the previous column, dynamically determining the search neighborhood of the current column, calculating the cost of the pixels in the search neighborhood, and selecting the pixel with the lowest cost as the center point of the current column to obtain two center lines;

[0062] S103, respectively extracting the inflection points on both sides of the weld groove based on the two center lines, and taking the midpoint of the inflection points on both sides as the estimated position of the weld center point;

[0063] S104, taking the estimated position of the weld center point of each frame of grayscale image as a node, and sequentially connecting the nodes in the welding time order to construct edges to obtain a graph structure;

[0064] In S105, the measured position of the center point of each grayscale image and the smoothness of the estimated position of the center point between adjacent grayscale images are used as constraint relationships, and the state values of the nodes are optimized by minimizing the constraint errors of all edges in the graph structure within the sliding window, and finally the smooth trajectory of the weld center point is determined for welding path planning and weld tracking.

[0065] In this embodiment, for the laser stripe image converted into a grayscale image, the center line is extracted successively by dynamic neighborhood search, and the estimated position of the weld center point is determined by inflection point extraction. Combining the graph structure and the sliding window for graph structure optimization, the smooth trajectory of the weld center point is determined, realizing high-precision tracking of various welds in a high-noise environment, taking into account real-time performance and global accuracy, and finally improving the tracking accuracy and stability of the welding robot system.

[0066] The weld tracking method based on neighborhood search and graph optimization in this embodiment can also be used for real-time welding deviation correction.

[0067] Next, in combination with Figure 2 , the specific implementation process of each step in the weld tracking method based on neighborhood search and graph optimization in the embodiment of the present invention will be given in detail.

[0068] In step S101, a line laser sensor installed at the end of the robotic arm can be used to collect the laser stripe image of the weld groove. If a laser stripe image containing the shape of the weld groove is collected, step S102 can be continued.

[0069] It should be noted here that in other embodiments, other existing image acquisition devices can also be used to collect the laser stripe image of the weld groove.

[0070] When the line laser sensor uses a color RGB camera instead of a grayscale camera, the color image is first converted into a grayscale image, and then the ROI area is selected to ensure that only one weld groove is included in the image, as Figure 5 shown.

[0071] The weld tracking method based on neighborhood search and graph optimization in the embodiment of the present invention does not need to traverse all the pixels of the grayscale image, and does not need to perform common denoising and image enhancement operations such as median / Gaussian filtering, contrast enhancement, and binarization on the entire image, thus saving computing resources.

[0072] In step S102, the two-way column-by-column pixel traversal is from left to right and from right to left.

[0073] After the two-way traversal is completed, the center lines obtained from the two traversals are stored in two arrays respectively.

[0074] Specifically, during the process of traversing the image column by column, if the distance between the position of the pixel with the maximum gray value at the leftmost / rightmost side of the current frame image and the position of the center point of the previous frame exceeds a set number (for example, 10) of pixels, it is determined that the selection of the center point is incorrect. At this time, the position of the previous frame is directly used as the center point position of the current frame.

[0075] Taking the traversal from left to right column by column as an example, since there is no position of the center point of the previous column as a reference for the leftmost column of pixels, it is necessary to traverse the entire column of pixels and select the pixel with the maximum gray value as the center point position of this column.

[0076] However, if the welding noise exactly appears in this column, its higher gray value may cover the gray value of the laser stripe, resulting in an incorrect selection of the center point, as Figure 6 shown. To avoid this situation, an error correction mechanism based on the position of the leftmost center point of the previous frame image is added: if the distance between the position of the pixel with the maximum gray value at the leftmost side of the current frame image and the position of the center point of the previous frame exceeds 10 pixels, it is determined that the selection of the center point is incorrect. At this time, the position of the previous frame is directly used as the center point position of the current frame. The reliability of this mechanism comes from the result of the center point extraction of the first frame image: in the initial stage of weld tracking, the first frame image has not started to arc, there is no welding noise, the image quality is the best, and the extracted center point position is accurate and can be used as a reference for subsequent frames. The traversal method from right to left is consistent with the above logic.

[0077] Center point extraction for non-initial columns. Adopt a dynamic neighborhood search strategy. When traversing the image column by column, based on the position of the center point extracted from the previous column , dynamically determine the search neighborhood of the current column:

[0078] ;

[0079] where r is the search radius, usually set according to the width of the weld feature and the noise level, generally taking 2 - 5.

[0080] For each pixel point p in the search neighborhood, calculate the cost function , to measure its suitability as the center point of the current column.

[0081] Select the pixel point with the minimum cost function from the search neighborhood as the center point of the current column :

[0082] ;

[0083] Specifically, during the process of traversing the image column by column, the expression for the cost of the pixels in the search neighborhood is:

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] wherein, is a weight parameter; is an expression of the cost of pixels in the search neighborhood; is the distance cost; is the gray - level cost; is the dynamic noise adjustment cost; is a pixel; is the center point of the previous column; is the gray - level value of the pixel point; is the deviation of the pixel point from the position change direction of the previous frame.

[0089] wherein, the distance cost represents the distance between the pixel point p and the center point of the previous column;

[0090] The gray - level cost represents the influence of the gray - level value of the pixel point on the selection of the center point. The lower the gray - level value, the higher the cost;

[0091] When the noise is strong, based on the moving direction of the center point of the previous frame relative to the previous column, the cost function is dynamically adjusted to reduce the pixel cost in the moving direction of the center point and increase the pixel cost in the noise direction.

[0092] In this embodiment, the cost function of pixels in the search neighborhood is constructed by comprehensively considering the distance cost, gray - level cost and dynamic noise adjustment cost, taking into account three aspects of factors: distance, gray - level and dynamic noise, and realizing high - precision tracking of multiple welds in a high - noise environment.

[0093] In step S102, strong noise situations and laser stripe interruption situations may also occur. The above - mentioned cost function can preferably select the center - point pixel from the search neighborhood of each column. Now, some special situations need to be processed, such as Figure 7 As shown, if the proportion of pixels with high gray - level values exceeds a certain threshold T, it indicates that the welding arc light noise has partially or completely blocked the laser stripe center line in the search neighborhood. At this time, instead of using the cost function to solve, the center - point pixel position of this column in this frame image is directly selected according to the moving direction of the center - point pixel of this column in the previous - frame image.

[0094] If the center line of the current - frame gray - level image is blocked by noise, the center - line shape of the previous - frame image not blocked by noise is used to fill in the current - frame image;

[0095] If the gray values of all pixels in the search neighborhood are less than the set threshold (the size of this threshold is adjusted by actual factors such as the reflection intensity of the workpiece surface and the camera exposure time), it is determined that the laser stripe is interrupted, and the processing continues with the next column of pixels; if the interruption occurs continuously for a set number of times (for example, 10 times), it is determined that the laser stripe is completely interrupted, the traversal is stopped, and the column coordinates of the center points of each subsequent column are set to -1.

[0096] In step S103, the dynamic fitting method is respectively used based on the two centerlines to extract the inflection points on both sides of the weld groove. The process is as follows:

[0097] Starting from the leftmost / rightmost end of the centerline, select the first set number of points, and use the least squares method to fit a straight line as the initial fitting result;

[0098] When traversing the subsequent points, judge the distance from the current point to the fitting straight line point by point; if the distance from the point to the straight line is less than the preset threshold, add this point to the current fitting point set; when the number of newly added points reaches the set update threshold, refit the straight line;

[0099] If a set number of consecutive points deviate from the fitting straight line, it is judged that the centerline bends at the current position, the fitting is stopped, and the current position is recorded as the inflection point.

[0100] Figure 4 In (a), (b), (c), and (d) of, the inflection point positions of four different weld grooves are given. In order to improve the efficiency and accuracy of inflection point detection, a dynamic fitting mechanism is introduced. The dynamic fitting improves the fitting accuracy by reducing the redundant calculations on the long straight line segments and enhances the robustness to complex weld shapes. The inflection point detection is based on the weld centerline, and the inflection point positions on the left and right sides of the weld groove are respectively extracted, and the midpoint of the inflection points on both sides is used as the weld center point.

[0101] The following takes the inflection point detection of traversing the centerline from left to right as an example:

[0102] Initial fitting: Starting from the leftmost end of the centerline, select the first 100 points, and use the least squares method to fit a straight line as the initial fitting result.

[0103] Dynamic update of the fitting straight line: When traversing the subsequent points, judge the distance from the current point to the fitting straight line point by point. If the distance from the point to the straight line is less than the threshold max_distance (the threshold is determined according to the surface quality of the welded part), add this point to the current fitting point set. When the number of newly added points reaches the set update threshold (such as 50 points), refit the straight line to further improve the fitting accuracy.

[0104] Inflection point detection: If 10 consecutive points deviate from the fitted line (the deviation distance is greater than max_distance), it is considered that the center line bends at this position, stop fitting, and record this position as the inflection point. It should be noted that even if the extracted center line is interrupted, it does not affect the inflection point detection here. The column coordinates of the center points at the interruption are all set to -1, which will surely deviate from the fitted line, so the discontinuous points will also be detected.

[0105] In step S104, the obtained graph structure is used for the graph structure optimization in the subsequent step 105.

[0106] In step S105, a graph optimization method is used to track the center points. The states and constraints are represented by component nodes and edges, and then a global cost function is minimized to solve the problem, so as to finally determine the position of the weld center point.

[0107] The cost function corresponding to the state value of the optimized node is:

[0108] ;

[0109] Among them, the first term is the error between the measured position and the estimated position of the center point of the grayscale image; the second term is the smoothness constraint of the estimated position of the center point between adjacent frames of grayscale images; is the measured position of the center point of the th grayscale image, is the estimated position of the center point of the th grayscale image; is the estimated position of the center point of the th grayscale image; is the weight of the smoothness constraint. represents the smoothness of the estimated position of the center point between adjacent frames of grayscale images. Specifically, a sliding window with a fixed size is maintained, and the measured positions of the center points of the last N frames are stored in the window.

[0110] In some alternative embodiments, Ceres Solver can be called to minimize the cost function through a non-linear least squares solver, and the optimized center point position is output for welding path planning and weld tracking. Among them, Ceres Solver is an efficient non-linear least squares optimization library for solving optimization problems with constraint relationships.

[0111] It can be understood here that in other embodiments, other existing methods can also be used to solve the cost function corresponding to the state value of the above optimized node.

[0112] Figure 3This is a schematic structural diagram of a weld seam tracking system based on neighborhood search and graph optimization in an embodiment of the present invention. This embodiment corresponds to the weld seam tracking method based on neighborhood search and graph optimization in Figure 1 As shown in Figure 3 the weld seam tracking system based on neighborhood search and graph optimization in this embodiment may include:

[0113] An image acquisition module 301, which is used to acquire the laser stripe image of the weld seam groove and convert it into a grayscale image;

[0114] A center line extraction module 302, which is used to traverse the grayscale image pixel by pixel in two directions by column; when traversing the image by column, based on the center point position extracted from the previous column, dynamically determine the search neighborhood of the current column, calculate the cost of the pixels in the search neighborhood, and select the pixel with the lowest cost as the center point of the current column to obtain two center lines;

[0115] A center point estimation module 303, which is used to extract the inflection points on both sides of the weld seam groove based on the two center lines respectively, and use the midpoint of the inflection points on both sides as the estimated position of the weld seam center point;

[0116] A graph structure construction module 304, which is used to use the estimated position of the weld seam center point of each frame of grayscale image as a node, and connect the nodes in sequence according to the welding time sequence to construct edges to obtain a graph structure;

[0117] A graph structure optimization module 305, which is used to use the measured position of the center point of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frames of grayscale images as constraint relations, and optimize the state values of the nodes by minimizing the constraint errors of all edges in the graph structure within the sliding window, and finally determine the smooth trajectory of the weld seam center point for welding path planning and weld seam tracking.

[0118] It should be noted here that Figure 3 each module in the weld seam tracking system based on neighborhood search and graph optimization in Figure 1 corresponds one by one to each step in the weld seam tracking method based on neighborhood search and graph optimization in

[0119] and their specific implementation processes are the same, so they will not be repeated here.

[0120] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a local area network (LAN) card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. A removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is installed on the drive as needed so that a computer program read therefrom is installed into the storage section as needed.

[0121] When the central processing unit in the electronic device of this embodiment executes the program, it implements the steps in the weld seam tracking method based on neighborhood search and graph optimization as Figure 1 shown.

[0122] Specifically, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing Figure 1 the method shown. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section, and / or installed from a removable medium. When the computer program is executed by the central processing unit, various functions defined in the device of the present application are executed.

[0123] Wherein, Figure 1 the computer program instructions corresponding to the method shown can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in one process Figure 1 or multiple processes and / or one block Figure 1 or multiple blocks.

[0124] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.

[0125] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A weld seam tracking method based on neighborhood search and graph optimization, characterized in that Including: Obtain a laser stripe image of the weld groove and convert it into a grayscale image; Traverse the grayscale image pixel by pixel in both directions column by column; when traversing the image column by column, based on the center point position extracted from the previous column, dynamically determine the search neighborhood of the current column, calculate the cost of the pixels in the search neighborhood, and select the pixel with the lowest cost as the center point of the current column to obtain two center lines; Extract the inflection points on both sides of the weld groove based on the two center lines respectively, and take the midpoint of the inflection points on both sides as the estimated position of the weld center point; Take the estimated position of the weld center point of each frame of grayscale image as a node, and connect the nodes in sequence according to the welding time sequence to construct edges to obtain a graph structure; Take the measured position of the center point of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frames of grayscale images as constraint relationships, and optimize the state value of the node by minimizing the constraint error of all edges in the graph structure within the sliding window, and finally determine the smooth trajectory of the weld center point for welding path planning and weld tracking.

2. The weld seam tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that During the process of traversing the image column by column, if the distance between the pixel position with the largest grayscale value on the leftmost / rightmost side of the current frame image and the center point position of the previous frame exceeds a set number of pixels, it is determined that the center point selection is incorrect. At this time, directly use the position of the previous frame as the center point position of the current frame.

3. The seam tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that, During the process of traversing the image column by column, the expression for the cost of the pixels in the search neighborhood is: ； ； ； ； Among them, is the weight parameter; is the expression of the cost of pixels in the search neighborhood; is the distance cost; is the gray-level cost; is the dynamic noise adjustment cost; is the pixel; is the center point of the previous column; is the gray-level value of the pixel point; is the deviation of the pixel point from the position change direction of the previous frame.

4. The seam tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that If during the traversal of the current frame of grayscale image, the proportion of the number of pixels with a set high grayscale value in the search neighborhood of a certain column exceeds the set threshold, it is determined that the welding arc light noise partially or completely obscures the laser stripe center line in the search neighborhood, and directly select the center point pixel position of the corresponding column of the current frame of grayscale image according to the moving direction of the center point pixel of the corresponding column of the previous frame of grayscale image.

5. The seam tracking method based on neighborhood search and graph optimization according to claim 1 or 4, characterized in that, If the center line of the current frame of grayscale image is obscured by noise, use the center line shape of the previous frame that was not obscured by noise to fill in the current frame image; Or if the grayscale values of all pixels in the search neighborhood are less than the set threshold, it is determined that the laser stripe is interrupted, and continue to process the pixels of the next column; if the interruption occurs continuously for a set number of times, it is determined that the laser stripe is completely interrupted, stop traversing, and set the column coordinate of the center point of each subsequent column to -1.

6. The seam tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that Based on the two center lines, respectively use the dynamic fitting method to extract the inflection points on both sides of the weld groove, and the process is as follows: Start from the leftmost / rightmost end of the center line, select the first set number of points, and use the least squares method to fit a straight line as the initial fitting result; When traversing the subsequent points, judge the distance from the current point to the fitting straight line point by point; if the distance from the point to the straight line is less than the preset threshold, add the point to the current fitting point set; when the number of newly added points reaches the set update threshold, refit the straight line; If a continuous set number of points deviate from the fitting straight line, it is determined that the center line bends at the current position, stop fitting, and record the current position as the inflection point.

7. The seam tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that The cost function corresponding to the state value of the optimized node is as follows: ； Among them, the first item is the error between the measured position and the estimated position of the center point of the grayscale image; the second item is the smoothness constraint of the estimated position of the center point between adjacent frame grayscale images. is the measured position of the center point of the -th grayscale image, is the estimated position of the center point of the -th grayscale image; is the estimated position of the center point of the -th grayscale image; is the weight of the smoothness constraint.

8. A weld seam tracking system based on neighborhood search and graph optimization, characterized in that, Including: An image acquisition module, which is used to obtain a laser stripe image of the weld groove and convert it into a grayscale image; A centerline extraction module, which is used to traverse the grayscale image pixel by pixel in both directions column by column; when traversing the image column by column, based on the center point position extracted from the previous column, dynamically determine the search neighborhood of the current column, calculate the cost of the pixels in the search neighborhood, and select the pixel with the lowest cost as the center point of the current column, obtaining two centerlines; A center point estimation module, which is used to extract the inflection points on both sides of the weld groove based on the two centerlines respectively, and use the midpoint of the inflection points on both sides as the estimated position of the weld center point; A graph structure construction module, which is used to use the estimated position of the weld center point of each frame of grayscale image as a node, and sequentially connect the nodes in the welding time order to construct edges, obtaining a graph structure; A graph structure optimization module, which is used to use the center point measurement position of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frames of grayscale images as constraint relationships, and optimize the state values of the nodes by minimizing the constraint errors of all edges in the graph structure within the sliding window, and finally determine the smooth trajectory of the weld center point for welding path planning and weld tracking.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps in the weld tracking method based on neighborhood search and graph optimization according to any one of claims 1-7.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the weld tracking method based on neighborhood search and graph optimization according to any one of claims 1-7.

Images