Welding seam tracking method and system based on neighborhood search and graph optimization

Abstract

The invention belongs to the technical field of welding visual sensors, and provides a welding seam tracking method and system based on neighborhood search and graph optimization in order to solve the problem that an existing welding seam tracking method is difficult to adapt to a high-dynamic-change welding seam track. The weld seam tracking method based on neighborhood search and graph optimization comprises the following steps: converting a laser stripe image of a weld seam groove into a grayscale image; the grayscale image is traversed bidirectionally according to column pixels, and two center lines are extracted; inflection points on the two sides of the weld groove are extracted on the basis of the two center lines and serve as the estimated positions of the weld center points; taking the estimated position of the weld center point of each frame of gray level image as a node, and connecting the nodes according to a welding time sequence in sequence to construct edges to obtain a graph structure; the smoothness of the central point measurement position of each frame of gray scale image and the central point estimation position between the adjacent frames of gray scale images is used as a constraint relation, the smooth track of the welding seam central point is determined through graph optimization, and high-precision real-time tracking of the welding seam central point is achieved in combination with a sliding window.

Description

Technical Field

[0001] The present invention belongs to the technical field of welding vision sensors, and particularly relates to a weld seam tracking method and system based on neighborhood search and graph optimization. Background Technique

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, with the rapid development of robot technology and computer vision technology, the application scope of welding robots in industrial production has become wider and wider, and the usage scenarios cover many aspects such as a large number of standardized components and small batch customized parts. At present, the operation control of welding robots still mainly relies on teaching programming and offline programming. In order to improve welding accuracy and programming efficiency, installing a line laser sensor at the end of the robotic arm for weld seam scanning and tracking has become an important development trend in the field of welding robots. The introduction of the line laser vision sensor provides a new solution for the real-time tracking of welding paths.

[0004] However, in the prior art, the efficiency of extracting the center point by completely traversing all column pixels of the image is low, and due to interferences such as arc light noise and workpiece surface reflection in the welding environment, the quality of the weld seam image collected by the sensor may deteriorate, resulting in low reliability of inflection point detection and it is difficult to achieve a balance between real-time performance and accuracy in a dynamic scenario. In addition, the existing Kalman filtering method has limited control over long-term cumulative errors and is difficult to adapt to highly dynamic weld seam trajectories. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a weld seam tracking method and system based on neighborhood search and graph optimization, which improves the robustness and efficiency of center point extraction through dynamic neighborhood search and cost function design, and realizes high-precision real-time tracking of the weld seam center point by combining sliding window optimization.

[0006] To achieve the above object, the present invention adopts the following technical solutions: The first aspect of the present invention provides a weld seam tracking method based on neighborhood search and graph optimization.

[0007] In one or more embodiments, a weld seam tracking method based on neighborhood search and graph optimization is provided, including: Obtain the laser stripe image of the weld groove and convert it into a grayscale image; Traverse the grayscale image bidirectionally by column pixels; when traversing the image by column, based on the center point position extracted from the previous column, dynamically determine the search neighborhood of the current column, calculate the cost of the pixels in the search neighborhood, and select the pixel with the lowest cost as the center point of the current column to obtain two center lines; Extract the inflection points on both sides of the weld groove based on two centerlines respectively, and take the midpoint of the inflection points on both sides as the estimated position of the weld center point; Take the estimated position of the weld center point of each frame of grayscale image as a node, and connect the nodes in sequence according to the welding time order to construct edges, obtaining a graph structure; Take the measured position of the center point of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frames of grayscale images as constraint relationships, and optimize the state values of the nodes by minimizing the constraint errors of all edges in the graph structure within the sliding window, finally determining the smooth trajectory of the weld center point for welding path planning and weld tracking.

[0008] As an implementation manner, during the process of traversing the image column by column, if the distance between the pixel position with the maximum grayscale value on the leftmost / rightmost side of the current frame image and the position of the center point of the previous frame exceeds a set number of pixels, it is determined that the selection of the center point is incorrect. At this time, directly use the position of the previous frame as the position of the center point of the current frame.

[0009] As an implementation manner, during the process of traversing the image column by column, the expression for the cost of searching for pixels in the neighborhood is: ; ; ; ; Among them, is the weight parameter; is the expression for the cost of searching for pixels in the neighborhood; is the distance cost; is the grayscale cost; is the dynamic noise adjustment cost; is the pixel; is the center point of the previous column; is the grayscale value of the pixel point; is the deviation of the pixel point from the direction of position change of the previous frame.

[0010] As an implementation manner, if during the traversal of the current frame of grayscale image, the proportion of the number of pixels with a set high grayscale value in the search neighborhood of a certain column exceeds the set threshold, it is determined that the welding arc light noise partially or completely obscures the laser stripe centerline in the search neighborhood, and directly select the pixel position of the center point of the corresponding column of the current frame of grayscale image according to the moving direction of the center point pixel of the corresponding column of the previous frame of grayscale image.

[0011] As an implementation manner, if the centerline of the current frame of grayscale image is obscured by noise, the centerline shape of the previous frame of image not obscured by noise is used to supplement the current frame of image; As an implementation, if the grayscale values of all pixels in the search neighborhood are less than the set threshold, it is determined that the laser stripe is interrupted, and the processing proceeds to the next column of pixels; if the interruption occurs continuously for the set number of times, it is determined that the laser stripe is completely interrupted, the traversal is stopped, and the column coordinates of the center points of each subsequent column are set to -1.

[0012] As an implementation, based on the two centerlines, a dynamic fitting method is respectively used to extract the inflection points on both sides of the weld groove, and the process is as follows: Starting from the leftmost / rightmost end of the centerline, select the first set number of points, and use the least squares method to fit a straight line as the initial fitting result; When traversing the subsequent points, judge the distance from the current point to the fitting straight line point by point; if the distance from the point to the straight line is less than the preset threshold, add the point to the current fitting point set; when the number of newly added points reaches the set update threshold, refit the straight line; If a continuous set number of points deviate from the fitting straight line, it is determined that the centerline bends at the current position, the fitting is stopped, and the current position is recorded as the inflection point.

[0013] As an implementation, optimize the cost function corresponding to the state value of the node It is: ; Among them, the first item is the error between the measured position and the estimated position of the center point of the grayscale image; the second item is the smoothness constraint of the estimated position of the center point between adjacent frame grayscale images; is the measured position of the center point of the th grayscale image, is the estimated position of the center point of the th grayscale image; is the estimated position of the center point of the th grayscale image; is the weight of the smoothness constraint.

[0014] The second aspect of the present invention provides a weld tracking system based on neighborhood search and graph optimization.

[0015] In one or more embodiments, a weld tracking system based on neighborhood search and graph optimization includes: An image acquisition module, which is used to acquire the laser stripe image of the weld groove and convert it into a grayscale image; A centerline extraction module, which is used to traverse the grayscale image pixel by pixel in both directions; when traversing the image column by column, based on the center point position extracted from the previous column, dynamically determine the search neighborhood of the current column, calculate the cost of the pixels in the search neighborhood, and select the pixel with the lowest cost as the center point of the current column to obtain two centerlines; A center point estimation module, which is configured to extract the inflection points on both sides of the weld groove based on two center lines respectively, and use the midpoint of the inflection points on both sides as the estimated position of the weld center point; A graph structure construction module, which is configured to use the estimated position of the weld center point of each frame of grayscale image as a node, and connect the nodes in sequence according to the welding time order to construct edges, thereby obtaining a graph structure; A graph structure optimization module, which is configured to use the measured position of the center point of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frames of grayscale images as constraint relationships, and optimize the state values of the nodes by minimizing the constraint errors of all edges in the graph structure within a sliding window, and finally determine the smooth trajectory of the weld center point for welding path planning and weld tracking.

[0016] The third aspect of the present invention provides a computer-readable storage medium.

[0017] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps in the weld tracking method based on neighborhood search and graph optimization as described above are implemented.

[0018] The fourth aspect of the present invention provides an electronic device.

[0019] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps in the weld tracking method based on neighborhood search and graph optimization as described above are implemented.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) For the laser stripe image converted into a grayscale image, the present invention sequentially extracts the center line through dynamic neighborhood search, determines the estimated position of the weld center point through inflection point extraction, combines the graph structure and the sliding window to optimize the graph structure, determines the smooth trajectory of the weld center point, realizes high-precision tracking of various welds in a high-noise environment, takes into account real-time performance and global accuracy, and finally improves the tracking accuracy and stability of the welding robot system.

[0021] (2) The present invention is no longer limited to identifying and extracting a single weld type, can accurately extract the weld center points of various typical welds (such as V-shaped, lap joint, fillet joint), realizes dynamic detection and accurate extraction of inflection points, and effectively responds to sudden changes in weld shapes.

[0022] (3) In order to solve the situation that a relatively high grayscale value may cover the grayscale value of the laser stripe, resulting in incorrect selection of the center point, the present invention adds an error correction mechanism based on the position of the leftmost center point of the previous frame image, thereby improving the accuracy of determining the weld center point.

[0023] (4) The present invention introduces a dynamic fitting mechanism, which extracts the inflection points on both sides of the weld groove through the dynamic fitting method, thereby improving the efficiency and accuracy of inflection point detection. By reducing redundant calculations on long straight line segments, the fitting accuracy is improved, and the robustness to complex weld shapes is enhanced.

[0024] (5) The present invention integrates distance cost, grayscale cost and dynamic noise adjustment cost to construct a cost function for pixels in the search neighborhood, taking into account the three factors of distance, grayscale and dynamic noise, and achieves high-precision tracking of various welds in a high-noise environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 It is a schematic flow chart of a weld tracking method based on neighborhood search and graph optimization according to an embodiment of the present invention; Figure 2 It is a specific process diagram of weld seam tracking based on neighborhood search and graph optimization according to an embodiment of the present invention; Figure 3 It is a schematic diagram of the structure of a weld tracking system based on neighborhood search and graph optimization according to an embodiment of the present invention; Figure 4 Schematic diagram of identification inflection points of different groove welds in an embodiment of the present invention; Figure 5 Schematic diagram of ROI intercepting a single weld groove image in an embodiment of the present invention; Figure 6 is a schematic diagram showing arc noise in the first column on the left side of a weld image according to an embodiment of the present invention; Figure 7 Schematic diagram of laser stripe interruption in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0029] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0030] Figure 1 FIG. is a schematic flow chart of a weld seam tracking method based on neighborhood search and graph optimization in an embodiment of the present invention. As Figure 1 shown, the weld seam tracking method based on neighborhood search and graph optimization in this embodiment may include: S101, obtaining a laser stripe image of a weld groove and converting it into a grayscale image; S102, traversing the grayscale image pixel by pixel in both directions by column; when traversing the image by column, based on the center point position extracted from the previous column, dynamically determining the search neighborhood of the current column, calculating the cost of the pixels in the search neighborhood, and selecting the pixel with the lowest cost as the center point of the current column to obtain two center lines; S103, extracting the inflection points on both sides of the weld groove based on the two center lines respectively, and taking the midpoint of the inflection points on both sides as the estimated position of the weld center point; S104, taking the estimated position of the weld center point of each frame of grayscale image as a node, and connecting the nodes in sequence according to the welding time sequence to construct edges to obtain a graph structure; S105, taking the measured position of the center point of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frames of grayscale images as constraint relations, and optimizing the state values of the nodes by minimizing the constraint errors of all edges in the graph structure within the sliding window, and finally determining the smooth trajectory of the weld center point for welding path planning and weld seam tracking.

[0031] In this embodiment, for the laser stripe image converted into a grayscale image, the center line is extracted through dynamic neighborhood search in sequence, the estimated position of the weld center point is determined through inflection point extraction, the graph structure is optimized by combining the graph structure and the sliding window, and the smooth trajectory of the weld center point is determined, realizing high-precision tracking of various weld seams in a high-noise environment, taking into account both real-time performance and global accuracy, and finally improving the tracking accuracy and stability of the welding robot system.

[0032] The weld seam tracking method based on neighborhood search and graph optimization in this embodiment can also be used for real-time welding deviation correction.

[0033] Next, in combination with Figure 2 , the specific implementation processes of each step in the weld seam tracking method based on neighborhood search and graph optimization in the embodiment of the present invention will be given in detail.

[0034] In step S101, a line laser sensor installed at the end of the robotic arm can be used to collect the laser stripe image of the weld groove. If a laser stripe image containing the shape of the weld groove is collected, step S102 can be continued.

[0035] It should be noted here that in other embodiments, other existing image acquisition devices can also be used to collect the laser stripe image of the weld groove.

[0036] When the line laser sensor uses a color RGB camera instead of a grayscale camera, the color image is first converted to a grayscale image, and then the ROI region is selected to ensure that only one weld groove is included in the image, as Figure 5 shown.

[0037] The weld tracking method based on neighborhood search and graph optimization according to the embodiment of the present invention does not need to traverse all the pixels of the grayscale image, and does not need to perform common denoising and image enhancement operations such as median / Gaussian filtering, contrast enhancement, and binarization on the entire image, thus saving computing resources.

[0038] In step S102, the two-way column pixel traversal is from left to right and from right to left.

[0039] After the two-way traversal is completed, the centerlines obtained from the two traversals are respectively stored in two arrays.

[0040] Specifically, during the process of traversing the image column by column, if the distance between the position of the pixel with the maximum grayscale value on the leftmost / rightmost side of the current frame image and the position of the center point of the previous frame exceeds a set number (for example, 10) of pixels, it is determined that the center point selection is incorrect. At this time, the position of the previous frame is directly used as the center point position of the current frame.

[0041] Taking the column-by-column traversal from left to right as an example, since the leftmost column of pixels lacks the position of the center point of the previous column as a reference, it is necessary to traverse the entire column of pixels and select the pixel with the maximum grayscale value as the center point position of this column.

[0042] However, if welding noise just appears in this column, its relatively high grayscale value may cover the grayscale value of the laser stripe, resulting in incorrect center point selection, as Figure 6As shown. To avoid this situation, an error correction mechanism based on the position of the leftmost center point of the previous frame image is added: If the distance between the position of the pixel with the largest gray value on the leftmost side of the current frame image and the center point position of the previous frame exceeds 10 pixels, it is determined that the center point selection is incorrect. At this time, the position of the previous frame is directly used as the center point position of the current frame. The reliability of this mechanism comes from the center point extraction result of the first frame image: In the initial stage of weld tracking, the first frame image has not started arc welding, there is no welding noise, the image quality is the best, and the extracted center point position is accurate and can be used as a reference for subsequent frames. The traversal method from right to left is consistent with the above logic.

[0043] Center point extraction for non-initial columns. Using a dynamic neighborhood search strategy, when traversing the image column by column, based on the center point position extracted from the previous column , dynamically determine the search neighborhood of the current column: ; where r is the search radius, usually set according to the width of the weld feature and the noise level, generally taking 2 - 5.

[0044] For each pixel point p in the search neighborhood, calculate the cost function , to measure its suitability as the center point of the current column.

[0045] Select the pixel point with the smallest cost function from the search neighborhood as the center point of the current column : ; Specifically, during the process of traversing the image column by column, the expression for the cost of the pixels in the search neighborhood is: ; ; ; ; Among them, is the weight parameter; is the expression for the cost of the pixels in the search neighborhood; is the distance cost; is the gray level cost; is the dynamic noise adjustment cost; is the pixel; is the center point of the previous column; is the gray level value of the pixel point; is the deviation of the pixel point from the position change direction of the previous frame.

[0046] Among them, the distance cost represents the distance between the pixel point p and the center point of the previous column Distance; Gray - level cost Indicates the influence of the gray - level value of a pixel on the selection of the center point. The lower the gray - level value, the higher the cost. When the noise is strong, based on the moving direction of the center point of the previous frame relative to the previous column, the cost function is dynamically adjusted to reduce the pixel cost in the moving direction of the center point and increase the pixel cost in the noise direction.

[0047] In this embodiment, the cost function of pixels in the search neighborhood is constructed by comprehensively considering the distance cost, gray - level cost, and dynamic noise adjustment cost, taking into account three aspects of factors: distance, gray - level, and dynamic noise, and realizing high - precision tracking of various welds in a high - noise environment.

[0048] In step S102, strong - noise situations and laser - stripe interruption situations may also occur. The above - mentioned cost function can preferably select the center - point pixels from the search neighborhood of each column. Now, some special situations need to be processed, such as Figure 7 As shown, if the proportion of pixels with high gray - level values exceeds a certain threshold T, it indicates that the welding arc light noise has partially or completely blocked the center line of the laser stripe in the search neighborhood. At this time, instead of using the cost function to solve, the center - point pixel position of this column in the current frame is directly selected according to the moving direction of the center - point pixel of this column in the previous - frame image.

[0049] If the center line of the current - frame gray - level image is blocked by noise, the shape of the center line of the previous - frame image not blocked by noise is used to fill in the current - frame image. If the gray - level values of all pixels in the search neighborhood are less than a set threshold (the size of this threshold is adjusted by actual factors such as the surface reflection intensity of the workpiece and the camera exposure time), it is determined that the laser stripe is interrupted, and the next column of pixels is continued to be processed; if the interruption occurs continuously for a set number of times (for example, 10 times), it is determined that the laser stripe is completely interrupted, the traversal is stopped, and the column coordinates of the center point of each subsequent column are set to - 1.

[0050] In step S103, based on the two center lines, a dynamic fitting method is used to extract the inflection points on both sides of the weld groove. The process is as follows: Starting from the left - most / right - most end of the center line, a set number of points are selected in advance, and the least - squares method is used to fit a straight line as the initial fitting result. When traversing the subsequent points, the distance from the current point to the fitting straight line is judged point - by - point; if the distance from the point to the straight line is less than a preset threshold, the point is added to the current fitting - point set; when the number of newly added points reaches the set update threshold, the straight line is refitted. If a set number of consecutive points deviate from the fitting straight line, it is judged that the center line bends at the current position, the fitting is stopped, and the current position is recorded as the inflection point.

[0051] Figure 4 Among (a), (b), (c) and (d), the inflection point positions of four different weld grooves are given. In order to improve the efficiency and accuracy of inflection point detection, a dynamic fitting mechanism is introduced. Dynamic fitting improves the fitting accuracy by reducing redundant calculations on long straight line segments and enhances the robustness to complex weld shapes. Inflection point detection is based on the weld center line. The inflection point positions on the left and right sides of the weld groove are extracted respectively, and the midpoint of the inflection points on both sides is used as the weld center point.

[0052] The following takes the inflection point detection of traversing the center line from left to right as an example: Initial fitting: Starting from the leftmost end of the center line, select the first 100 points and fit a straight line using the least squares method as the initial fitting result.

[0053] Dynamic update of the fitting straight line: When traversing the subsequent points, judge the distance from the current point to the fitting straight line point by point. If the distance from the point to the straight line is less than the threshold max_distance (the threshold is determined according to the surface quality of the welded part), add the point to the current fitting point set. When the number of newly added points reaches the set update threshold (such as 50 points), refit the straight line to further improve the fitting accuracy.

[0054] Inflection point detection: If 10 consecutive points deviate from the fitting straight line (the deviation distance is greater than max_distance), it is considered that the center line bends at this position, stop fitting, and record this position as an inflection point. It should be noted that even if the extracted center line is interrupted, it does not affect the inflection point detection here. The column coordinates of the center points at the interruption are all set to -1 and will definitely deviate from the fitted straight line, so the discontinuity points will also be detected.

[0055] In step S104, the obtained graph structure is used for the graph structure optimization in the subsequent step 105.

[0056] In step S105, a graph optimization method is used to track the center point. The state and constraints are represented by component nodes and edges, and then a global cost function is minimized to solve the problem, so as to finally determine the weld center point position.

[0057] The cost function corresponding to the state value of the optimized node is: ; Among them, the first term is the error between the measured position and the estimated position of the center point of the grayscale image; the second term is the smoothness constraint of the estimated position of the center point between adjacent frame grayscale images; is the measured position of the center point of the th grayscale image, is the The estimated position of the center point of a grayscale image; is the estimated position of the center point of the th grayscale image; is the weight of the smoothness constraint. represents the smoothness of the estimated position of the center point between adjacent frame grayscale images. Specifically, a sliding window of a fixed size is maintained, and the measured positions of the center points of the nearest N frames are stored within the window.

[0058] In some alternative embodiments, the Ceres Solver can be called to minimize the cost function through a non - linear least - squares solver, and the optimized center point position can be output for welding path planning and weld seam tracking. Among them, the Ceres Solver is an efficient non - linear least - squares optimization library used to solve optimization problems with constraint relationships.

[0059] It can be understood here that in other embodiments, other existing methods can also be used to solve the cost function corresponding to the state values of the above - mentioned optimization nodes.

[0060] Figure 3 is a schematic structural diagram of a weld seam tracking system based on neighborhood search and graph optimization in an embodiment of the present invention. This embodiment corresponds to the Figure 1 weld seam tracking method based on neighborhood search and graph optimization. As Figure 3 shown, the weld seam tracking system based on neighborhood search and graph optimization in this embodiment may include: An image acquisition module 301, which is used to acquire the laser stripe image of the weld groove and convert it into a grayscale image; A centerline extraction module 302, which is used to traverse the grayscale image pixel - by - pixel in both directions by column; when traversing the image by column, based on the center point position extracted from the previous column, the search neighborhood of the current column is dynamically determined, the cost of the pixels within the search neighborhood is calculated, and the pixel with the lowest cost is selected as the center point of the current column, obtaining two centerlines; A center point estimation module 303, which is used to extract the inflection points on both sides of the weld groove based on the two centerlines respectively, and take the mid - point of the inflection points on both sides as the estimated position of the weld center point; A graph structure construction module 304, which is used to use the estimated position of the weld center point of each frame of grayscale image as a node, and connect the nodes in sequence according to the welding time sequence to construct edges, obtaining a graph structure; A graph structure optimization module 305, which is used to use the measured position of the center point of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frame grayscale images as constraint relationships, and optimize the state values of the nodes by minimizing the constraint errors of all edges in the graph structure within the sliding window, and finally determine the smooth trajectory of the weld center point for welding path planning and weld seam tracking.

[0061] It should be noted here that Figure 3 each module in the weld seam tracking system based on neighborhood search and graph optimization in Figure 1 corresponds one by one to each step in the weld seam tracking method based on neighborhood search and graph optimization in

[0062] In one or more embodiments, an electronic device includes a central processing unit (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage section into a random access memory (RAM). In the RAM, various programs and data required for system operation are also stored. The central processing unit, ROM, and RAM are connected to each other via a bus 404. An input / output (I / O) interface is also connected to the bus.

[0063] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a local area network (LAN) card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. A removable medium, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive as needed so that a computer program read from it can be installed into the storage section as needed.

[0064] When the central processing unit in the electronic device of this embodiment executes the program, it implements the steps in the weld seam tracking method based on neighborhood search and graph optimization as Figure 1 shown.

[0065] Specifically, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing Figure 1 the method shown. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section, and / or installed from a removable medium. When the computer program is executed by the central processing unit, it executes various functions defined in the device of the present application.

[0066] Among them, Figure 1The computer program instructions corresponding to the methods shown can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device implements the processes Figure 1 one process or multiple processes and / or blocks Figure 1 the functions specified in one block or multiple blocks.

[0067] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM), or a random access memory (RAM), etc.

[0068] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A weld tracking method based on neighborhood search and graph optimization, characterized in that: include: Obtain the laser stripe image of the weld groove and convert it into a grayscale image; Bidirectionally traversing the grayscale image by columns of pixels; when traversing the image by columns, dynamically determining the search neighborhood of the current column based on the center point position extracted from the previous column, calculating the cost of the pixels in the search neighborhood, selecting the pixel with the lowest cost as the center point of the current column, and obtaining two center lines; Based on the two center lines, the inflection points on both sides of the weld groove are extracted respectively, and the midpoints of the inflection points on both sides are used as the estimated position of the weld center point; The estimated position of the weld center point of each frame of the grayscale image is taken as a node, and the nodes are connected in sequence according to the welding time sequence to construct edges to obtain a graph structure; The measured position of the center point of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frames of grayscale images are taken as constraint relationships. The state value of the node is optimized by minimizing the constraint error of all edges in the graph structure within the sliding window, and finally the smooth trajectory of the center point of the weld is determined for welding path planning and weld tracking.

2. The weld tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that: When traversing the image by column, if the distance between the pixel position with the largest grayscale value on the left / right side of the current frame image and the center point position of the previous frame exceeds the set number of pixels, it is judged that the center point selection is wrong. In this case, the position of the previous frame is directly used as the center point position of the current frame.

3. The weld tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that: In the process of traversing the image column by column, the cost of searching for pixels in the neighborhood is expressed as: ； ； ； ； in, is the weight parameter; is the expression for the cost of searching pixels in the neighborhood; For the cost of distance; is the grayscale cost; Adjust the cost for dynamic noise; is pixel; is the center point of the previous column; is the gray value of the pixel; It is the deviation of the pixel point from the previous frame position change direction.

4. The weld tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that: If, during the traversal of the grayscale image of the current frame, the proportion of pixels with a set high grayscale value in the search neighborhood of a certain column exceeds a set threshold, it is judged that the welding arc noise partially or completely blocks the center line of the laser stripe in the search neighborhood, and the center point pixel position of the corresponding column of the grayscale image of the current frame is directly selected according to the moving direction of the center point pixel of the corresponding column of the grayscale image of the previous frame.

5. The weld tracking method based on neighborhood search and graph optimization as claimed in claim 1 or 4, characterized in that: If the center line of the current frame grayscale image is blocked by noise, the center line shape of the previous frame image that is not blocked by noise is used to fill in the current frame image; Or if the grayscale values ​​of all pixels in the search neighborhood are less than the set threshold, the laser stripe is judged to be interrupted and the next column of pixels is processed; if the interruption is set for a continuous number of times, the laser stripe is judged to be completely interrupted, the traversal is stopped, and the column coordinates of the center point of each subsequent column are set to -1.

6. The weld tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that: Based on the two center lines, the dynamic fitting method is used to extract the inflection points on both sides of the weld groove. The process is as follows: Starting from the leftmost / rightmost end of the center line, select a preset number of points and use the least squares method to fit a straight line as the initial fitting result; When traversing subsequent points, determine the distance from the current point to the fitted line point by point; if the distance from the point to the line is less than the preset threshold, add the point to the current set of fitted points; when the number of newly added points reaches the set update threshold, refit the line; If the set number of points deviate from the fitting straight line continuously, it is judged that the center line is bent at the current position, the fitting is stopped, and the current position is recorded as the inflection point.

7. The weld tracking method based on neighborhood search and graph optimization according to claim 1, characterized in that: Optimize the cost function corresponding to the state value of the node for: ； Among them, the first term is the error between the measured position and the estimated position of the center point of the grayscale image; the second term is the smoothness constraint of the estimated position of the center point between adjacent frame grayscale images; For the The measured position of the center point of the grayscale image, For the The estimated position of the center point of a grayscale image; For the The estimated position of the center point of a grayscale image; is the weight of the smoothness constraint.

8. A weld tracking system based on neighborhood search and graph optimization, characterized in that: include: An image acquisition module, which is used to acquire a laser stripe image of a weld groove and convert it into a grayscale image; A centerline extraction module is used to bidirectionally traverse the grayscale image by columns of pixels; when traversing the image by columns, based on the center point position extracted from the previous column, the search neighborhood of the current column is dynamically determined, the cost of the pixels in the search neighborhood is calculated, and the pixel with the lowest cost is selected as the center point of the current column to obtain two centerlines; A center point estimation module is used to extract the inflection points on both sides of the weld groove based on the two center lines, and use the midpoints of the inflection points on both sides as the estimated position of the weld center point; A graph structure building module, which is used to use the estimated position of the weld center point of each frame of grayscale image as a node, connect the nodes in sequence according to the welding time sequence to build edges, and obtain a graph structure; A graph structure optimization module is used to use the measured position of the center point of each frame of grayscale image and the smoothness of the estimated position of the center point between adjacent frames of grayscale images as constraint relationships, optimize the state value of the node by minimizing the constraint errors of all edges in the graph structure within the sliding window, and finally determine the smooth trajectory of the center point of the weld for welding path planning and weld tracking.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the weld tracking method based on neighborhood search and graph optimization as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the weld tracking method based on neighborhood search and graph optimization as described in any one of claims 1 to 7 are implemented.

Images