Weld joint recognition method and device based on simulated structured light image

Abstract

The application discloses a kind of based on simulation structured light image's weld seam identification method and device, method is: obtaining weld seam image and pre-processing to obtain pre-processing image;Equal-interval collection column coordinate group is set and searches gray vertex as laser stripe candidate point;The width of laser stripe candidate point is calculated, and the laser stripe candidate point beyond the set range is filtered;Using simulation workstation to the weld seam image is simulated in real time to obtain simulation image;On pre-processing image is divided to obtain candidate point block;While on simulation image is divided to obtain simulation block;The similarity degree of the characteristic probability distribution of candidate point block and corresponding simulation block is measured to select actual laser stripe point;According to actual laser stripe point, least square method and random sampling consistency method are used to carry out line segment fitting, and according to weld seam interface type, the position point of weld seam is obtained.The application is based on visual simulation information design structured light weld seam positioning method, greatly improve the anti-interference ability, and the adaptability is stronger.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of structured light image processing and weld seam identification, and particularly relates to a weld seam identification method and device based on simulated structured light images. BACKGROUND

[0002] With the development of intelligent manufacturing, non-contact structured light vision sensors are increasingly widely used in the industry, especially in the fields of object scanning reconstruction, welding defect detection, and weld seam tracking. The vision sensor using the line structured light method meets the laser triangulation measurement model and is a non-contact, fast measurement, and high-precision measurement method. The vision sensor projects a laser line onto the surface of the measured object to form a laser stripe. Due to the different geometric shapes of the measured object surface, the laser stripe images are different. Then, the position of the laser stripe is extracted by image processing of different laser stripe images, and the spatial position of the point on the laser stripe is calculated according to the laser triangulation measurement model to extract the structural information of the surface of the measured object.

[0003] There are many existing processing methods for structured light stripe images. The processing steps generally include image preprocessing (filtering, smoothing, enhancement, etc.), image segmentation, edge extraction, and feature extraction. In each step, various methods can be used, such as Gaussian filtering, Gabor filtering for image filtering, and Canny edge extraction, gray centroid method for center line extraction. However, the existing methods are usually applied to the processing of weld seam images with weak reflection interference. However, in the case of strong reflection interference or when the gray scale distribution pattern is similar to the actual effective diffuse laser stripe, these methods are prone to failure, resulting in identification deviation. SUMMARY

[0004] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a weld seam identification method and device based on simulated structured light images, which can accurately extract the center line of the effective laser stripe in the presence of image interference and improve the application range of the structured light vision sensor.

[0005] To achieve the above purpose, the present application provides a weld seam identification method based on simulated structured light images, which comprises the following steps:

[0006] Obtain a weld seam image and preprocess it using the Otsu threshold method and morphological closing operation to obtain a preprocessed image;

[0007] Set an equidistant collection column coordinate group, and search for the gray scale vertex of the preprocessed image along the column direction of the gray scale distribution as the laser stripe candidate point;

[0008] Calculate the width of the laser stripe candidate point and filter out the laser stripe candidate points that exceed the set range;

[0009] The weld seam image is simulated in real time using a simulation workstation to obtain the simulation image;

[0010] Candidate point blocks are obtained by dividing the preprocessed image into blocks centered on the filtered laser stripe candidate points and with a set width as the edge; at the same time, simulation blocks are obtained by dividing the simulation image into blocks centered on the filtered laser stripe candidate points and with a set width as the edge.

[0011] Based on the Bach distance, the similarity of the feature probability distribution of candidate point blocks and corresponding simulated blocks in the same column of the column coordinate group is measured, and the center of the candidate point block with the highest similarity in each column is selected as the actual laser stripe point.

[0012] Based on the actual laser stripe points, line segments are fitted using the least squares method and random sampling consistency method, and the weld location points are obtained according to the weld interface type.

[0013] As a preferred technical solution, the equally spaced acquisition column coordinate group is represented as follows:

[0014] c j =L*j (j=1,2…k)

[0015] Where L is the set collection distance, j is the collection sequence number, and k is the maximum number of collections;

[0016] Let f(i) be the gray-level distribution of the column coordinate group collected at equal intervals, where i is the row coordinate. Perform gray-level vertex search on the preprocessed image along the column direction of the gray-level distribution f(i). Pixels that satisfy the following conditions are selected as candidate points for laser stripes:

[0017] a: Within a window centered at pixel i and with a set width of w, f(i) is the maximum value and f(i) is greater than the threshold T;

[0018] b: In Within the window, the number of pixels n with a positive or zero gradient. l :

[0019] c: In Within the window, the number of pixels n with negative or zero gradients. r :

[0020] Where η∈(0,1) is the proportionality coefficient.

[0021] As a preferred technical solution, the filtering of laser stripe candidate points exceeding the set range specifically involves:

[0022] The width W(i) of each candidate laser stripe point is obtained by the following formula:

[0023] W(i)=n l +n r

[0024] Where, n l for The number of pixels within the window whose gradient is positive or zero; n r for The number of pixels within the window whose gradient is negative or zero;

[0025] Determine whether the width W(i) of each candidate laser stripe point belongs to [W min W max If a point does not belong to the specified category, remove that candidate point from the laser stripe. min and W max The calculation formula is:

[0026] W min =μ-3σ,W max =μ+3σ

[0027]

[0028] Where μ is the average width of all candidate laser stripes, σ is the standard deviation of the width of all candidate laser stripes, and n is the number of candidate laser stripes.

[0029] As a preferred technical solution, the step of selecting the candidate laser stripe point with the highest similarity in each column as the actual laser stripe point specifically involves:

[0030] Calculate the feature probability distribution of the candidate point blocks and the corresponding simulation blocks respectively;

[0031] Based on the feature probability distribution, the similarity is obtained by calculating the Bach distance between the candidate point block and the corresponding simulation block;

[0032] The center of the candidate point block with the highest similarity is selected as the actual laser stripe point.

[0033] As a preferred technical solution, the formula for calculating the feature probability distribution is:

[0034]

[0035]

[0036] in, These are the probability density functions for the simulation block and the corresponding candidate point block, respectively. Let δ represent the probability features of the simulated block and the corresponding candidate point block under the histogram of feature u, respectively. δ is the Kronecker function, b is the grouping index of the pixel in the block in the histogram, m is the number of groups in the histogram of feature u, and x...i For the coordinates of the i-th pixel point in the simulation block, x is the center position of the simulation block, n represents the number of pixel points in the candidate point block, then For the coordinates of the i-th pixel point in the candidate point block, C is the normalization coefficient of the candidate point block, k is the kernel function of the probability density calculation method based on kernel density estimation, n h For the number of pixel points in the simulation block, C h For the normalization coefficient of the simulation block;

[0037] The calculation formula of the Bhattacharyya distance is:

[0038]

[0039]

[0040] Where B(p,q) is the Bhattacharyya distance of the feature probability distribution of the candidate point block and the corresponding simulation block in the same column of the column coordinate group, u is the feature, n u For the number of features, For the feature probability distribution of the candidate point block, For the feature probability distribution of the corresponding simulation block.

[0041] As a preferred technical solution, the line segment fitting is performed by using the least squares method and the random sample consensus method, specifically:

[0042] Randomly select two actual laser fringe points A and B and connect them into a straight line model AB, and define ε as the threshold value meeting the straight line model;

[0043] Calculate the distance of all other actual laser fringe points to the straight line model AB, and the actual laser fringe points with a distance less than or equal to ε are regarded as inliers, and the actual laser fringe points with a distance greater than ε are regarded as outliers;

[0044] When all the actual laser fringe points are calculated, calculate the ratio η of the number of inliers to the total number of actual laser fringe points, and when η is less than the set proportion threshold η0, the calculation is completed; and the precise parameters of the straight line model AB are obtained by fitting all the inliers using the least squares method;

[0045] If η is greater than the set proportion threshold η0, then two actual laser fringe points are reselected and connected as a new straight line model, and the above operations are repeated.

[0046] As a preferred technical solution, the weld joint type includes a connected type and a non-connected type;

[0047] When the weld joint type of the weld image is the connected type, then the straight line fitting intersection method is used to obtain the weld position point, specifically:

[0048] Obtaining the point coordinate (x0, y0) closest to the simulated weld point from the simulated image;

[0049] Obtaining two laser fitting line segments closest to the simulated weld point in the line segment fitting result, denoted as y=k1x+b1 and y=k2x+b2 respectively, and solving the intersection point of the two laser fitting line segments as the weld position point;

[0050] When the weld interface type of the weld image is a non-connection type, the feature points are extracted first and then the weld position point is calculated, specifically:

[0051] Obtaining the leftmost and rightmost actual laser stripe point coordinates (x1, y1) and (x2, y2) as search points;

[0052] Calculating the gradient direction of the search point Rotating the search direction by 90 degrees according to the gradient direction

[0053] Obtaining a test point by advancing a distance L along the search direction, if the gradient directions of the test point within the 8 adjacent pixels are all less than the threshold value σ, the test point is taken as the search point for further searching, otherwise it is an error point, the search is stopped and the previous search point of the error point is taken as the target feature point;

[0054] Taking the average of the two target feature points as the weld position point.

[0055] On the other hand, the present application provides a weld recognition system based on a simulated structured light image, which is applied to the weld recognition method based on the simulated structured light image, and comprises an image preprocessing module, a candidate point searching module, a candidate point filtering module, a simulation module, an image blocking module, a similarity calculation module and a weld position acquisition module.

[0056] The image preprocessing module is used for obtaining a weld image and pre-processing the weld image by using the Otsu threshold method and morphological closing operation to obtain a pre-processed image;

[0057] The candidate point searching module is used for setting an equidistant collection column coordinate group and searching for a gray vertex in the column direction of the gray distribution of the pre-processed image as a laser stripe candidate point;

[0058] The candidate point filtering module is used for calculating the width of the laser stripe candidate point and filtering the laser stripe candidate points beyond the set range;

[0059] The simulation module is used for using a simulation workstation to simulate the weld image in real time to obtain a simulated image;

[0060] The image block module is configured to center the filtered laser stripe candidate points on the preprocessed image, set a width as a side, and block to obtain candidate point blocks; and center the filtered laser stripe candidate points on the simulation image, set a width as a side, and block to obtain simulation blocks.

[0061] The similarity calculation module is configured to measure the similarity degree of the feature probability distribution of the candidate point blocks and the corresponding simulation blocks in the same column in the column coordinate group based on the Bhattacharyya distance, and select the center of the candidate point block with the largest similarity degree in each column as the actual laser stripe point.

[0062] The weld position acquisition module is configured to perform line segment fitting on the actual laser stripe point by using the least square method and the random sample consensus method, and obtain the weld position point according to the weld joint type.

[0063] In another aspect, the present application provides an electronic device, comprising:

[0064] at least one processor; and a memory connected to the at least one processor in communication; wherein,

[0065] The memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned weld identification method based on the simulation structured light image.

[0066] In another aspect, the present application provides a computer readable storage medium storing a program, wherein the program is executed by a processor to implement the above-mentioned weld identification method based on the simulation structured light image.

[0067] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0068] The present application is based on visual simulation information, and designs a method for removing interference and a method for positioning a weld, thereby realizing accurate positioning of the weld; in the weld identification process of the present application, the simulation image information is used to exclude interference to obtain the actual laser stripe position in view of the problem of similar laser stripe generation caused by strong reflection interference; in general, the structured light weld positioning method of the present application introduces the simulation laser stripe information given by visual simulation, greatly improves the anti-interference ability, has strong adaptability, and expands the application working range of the structured light visual sensor. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0070] Figure 1 The flow chart of the weld recognition method based on the simulated structured light image in the embodiments of the present application.

[0071] Figure 2 The weld image containing laser stripes in the embodiments of the present application.

[0072] Figure 3 The image after pre-processing in the embodiments of the present application.

[0073] Figure 4 The result image of searching the laser stripe candidate points in the embodiments of the present application.

[0074] Figure 5 The simulated workstation and simulated image in the embodiments of the present application.

[0075] Figure 6 The result image of the similarity comparison in the embodiments of the present application.

[0076] Figure 7 The weld position point image obtained after feature extraction in the embodiments of the present application.

[0077] Figure 8 The structure diagram of the weld recognition system based on the simulated structured light image in the embodiments of the present application.

[0078] Figure 9 The structure diagram of an electronic device in the embodiments of the present application. DETAILED DESCRIPTION

[0079] In order to make the person skilled in the art better understand the present application, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without any creative effort are within the protection scope of the present application.

[0080] Reference to an "embodiment" in this application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. As will be apparent to those of ordinary skill in the art, embodiments described in this application can be combined with one another.

[0081] As shown in Figure 1 , the embodiment based on the weld seam recognition method of the simulated structured light image includes the following steps:

[0082] S1, acquire a weld seam image and use Otsu thresholding and morphological closing operation for preprocessing to obtain a preprocessed image;

[0083] S2, set an equidistantly collected column coordinate group, and search for a gray scale vertex along the column direction of the gray scale distribution of the preprocessed image as a laser stripe candidate point;

[0084] S3, calculate the width of the laser stripe candidate point, and filter the laser stripe candidate points that exceed the set range;

[0085] S4, use a simulation workstation to perform real-time simulation on the weld seam image to obtain a simulated image;

[0086] S5, divide the preprocessed image into blocks with the filtered laser stripe candidate points as the center and a set width as the edge to obtain candidate point blocks; at the same time, divide the simulated image into blocks with the filtered laser stripe candidate points as the center and a set width as the edge to obtain simulated blocks;

[0087] S6, based on the Bath distance, measure the similarity degree of the feature probability distribution of the candidate point blocks and the corresponding simulated blocks in the same column in the column coordinate group, and select the center of the candidate point block with the largest similarity degree in each column as the actual laser stripe point;

[0088] S7, according to the actual laser stripe point, use the least square method and the random sample consensus method for line segment fitting, and obtain the weld seam position point according to the weld seam interface type.

[0089] When acquiring a weld seam image, due to the high smoothness of the surface of the measured object, the laser projected onto the surface of the measured object or the laser stripe reflection occurs, resulting in multiple white bright stripes in the weld seam image, as shown in Figure 2 . Therefore, the application is preprocessed, the laser stripe and the background are separated according to the adaptive threshold of the interclass variance size using Otsu thresholding, and the morphological closing operation is used to enhance the connectivity of the laser stripe. The processed image is shown in Figure 3 , which effectively separates the image background and the laser stripe.

[0090] Furthermore, candidate points are searched on the laser stripe using a predefined set of equally spaced acquisition column coordinates; wherein, the equally spaced acquisition column coordinates are represented as:

[0091] c j =L*j (j=1,2…k)

[0092] Where L is the set collection distance, j is the collection sequence number, and k is the maximum number of collections;

[0093] Let f(i) be the gray-level distribution of the column coordinate group collected at equal intervals, where i is the row coordinate. Perform gray-level vertex search on the preprocessed image along the column direction of the gray-level distribution f(i). Pixels that satisfy the following conditions are selected as candidate points for laser stripes:

[0094] a: Within a window of width w centered at i, f(i) is a maximum value and f(i) is greater than the threshold T;

[0095] b: In Within the window, the number of pixels n with a positive or zero gradient. l :

[0096] c: In Within the window, the number of pixels n with negative or zero gradients. r :

[0097] Where η∈(0,1) is the scaling factor. Here, 'a' is used for grayscale value comparison, requiring the pixel to be the maximum value within the window and greater than a set threshold; after satisfying this condition, gradient calculation is performed to determine if the window region satisfies conditions b and c. If both are satisfied, the pixel is considered a candidate laser stripe point. The searched candidate laser stripe points are as follows: Figure 4 As shown.

[0098] like Figure 5 As shown, in this embodiment, a simulation workstation is used to perform real-time simulation of the weld seam image to obtain the corresponding simulation image. The general working principle of the simulation workstation is as follows: First, the models and parameters of the welding robot and positioner, welding tool information (including welding torch and sensors), and welding workpiece model are imported; then, the actual pose of the welding robot is obtained in real time through TCP communication in the onboard simulation software and simulated in the simulation space. At the same time, the welding process data of the workpiece is collected through sensors, and the simulation image can be obtained by simulating in the simulation space.

[0099] Furthermore, the candidate laser stripe points obtained from the search are filtered to remove interference points, specifically as follows:

[0100] The width W(i) of each candidate laser stripe point is obtained by the following formula:

[0101] W(i)=n l +n r

[0102] Where, n l for The number of pixels within the window whose gradient is positive or zero; n r for The number of pixels within the window whose gradient is negative or zero;

[0103] Determine whether the width W(i) of each candidate laser stripe point belongs to [W min W max If a point does not belong to the specified category, then that candidate point of the laser stripe is removed, i.e., interference points are removed. min and W max The calculation formula is:

[0104] W min =μ-3σ,W max =μ+3σ

[0105]

[0106] Where μ is the average width of all candidate laser stripes, σ is the standard deviation of the width of all candidate laser stripes, and n is the number of candidate laser stripes.

[0107] Furthermore, after obtaining the simulation blocks in step S5, the actual laser stripe points are selected by calculating the Bach distance, specifically as follows:

[0108] First, calculate the feature probability distribution of the candidate point blocks and the corresponding simulation blocks, respectively. The calculation formula is as follows:

[0109]

[0110]

[0111] in, These are the probability density functions for the simulation block and the corresponding candidate point block, respectively. Let δ and b represent the probability features of the simulation block and the corresponding candidate point block under the histogram of feature u, respectively. δ is the Kronecker function, b is the grouping index of the pixel in the block under the histogram (when the input is a simulation block, it is the grouping index of the pixel in the simulation block under the histogram of feature u; if it is a candidate point block, it is the grouping index of the pixel in the candidate point block under the histogram of feature u), m is the number of groups in the histogram of feature u, and x... i Let x be the coordinate of the i-th pixel in the simulation block, x be the center position of the simulation block, and n be the number of pixels in the candidate point block. is the coordinate of the i th pixel point in the candidate point patch, C is the normalization coefficient of the candidate point patch, k is the kernel function of the probability density calculation method based on kernel density estimation, n h is the number of pixel points in the simulation patch, C h is the normalization coefficient of the simulation patch.

[0112] Then, based on the feature probability distribution, the Bhattacharyya distance between the candidate point patch and the corresponding simulation patch is calculated to obtain the similarity degree; wherein the calculation formula of the Bhattacharyya distance is:

[0113]

[0114]

[0115] wherein B (p, q) is the Bhattacharyya distance of the feature probability distribution of the candidate point patch and the corresponding simulation patch in the same column of the column coordinate group, u is the feature value, n u is the number of features, is the feature probability distribution of the candidate point patch, is the feature probability distribution of the corresponding simulation patch.

[0116] Finally, the center of the candidate point patch with the largest similarity degree, i.e. the center of the candidate point patch with the smallest Bhattacharyya distance, is selected as the actual laser stripe point, as Figure 6 shown.

[0117] Further, according to the obtained actual laser stripe point, the least squares method and the random sample consensus method are used for line segment fitting, specifically:

[0118] Randomly select two actual laser stripe points A and B and connect them into a straight line model AB, and define ε as a threshold value that meets the straight line model;

[0119] Calculate the distance of all other actual laser stripe points to the straight line model AB, and the actual laser stripe points with a distance less than or equal to ε are regarded as inliers, and the actual laser stripe points with a distance greater than ε are regarded as outliers;

[0120] When all the actual laser stripe points are calculated, calculate the ratio η of the number of inliers to the total number of actual laser stripe points, and when η is less than a set proportion threshold η0, the calculation is completed; the precise parameters of the straight line model AB are obtained by fitting all the inliers using the least squares method;

[0121] If η is greater than the set proportion threshold η0, then two actual laser stripe points are reselected and connected as a new straight line model, and the above operations are repeated.

[0122] After the line segment fitting, the weld position point is obtained according to the weld joint type, since the weld joint type includes the connecting type and the non-connecting type, thus the cases are divided; when the weld joint type of the weld image is the connecting type, then the straight line fitting intersection method is used to obtain the weld position point, specifically as follows:

[0123] The simulation weld point coordinate (x0, y0) closest to the simulation image is obtained;

[0124] The two laser fitting line segments closest to the simulation weld point in the line segment fitting result are obtained, and are respectively represented as y=k1x+b1 and y=k2x+b2, and the intersection of the two laser fitting line segments is solved as the weld position point; as shown in the following formula: Figure 7

[0125] When the weld joint type of the weld image is the non-connecting type, the weld feature points are often two endpoints of the fracture, thus the feature points need to be extracted first and then the weld position point is further calculated, specifically as follows:

[0126] The leftmost and rightmost actual laser stripe point coordinates (x1, y1) and (x2, y2) are obtained as search points;

[0127] The gradient direction of the search point is calculated The search direction is obtained by rotating 90 degrees according to the gradient direction

[0128] The test point is obtained by advancing a distance L along the search direction, if the gradient directions within 8 pixels adjacent to the test point and the gradient direction of the search point are all less than a threshold value σ, then the test point is taken as the search point to continue searching, otherwise it is an error point, the searching is stopped and the previous search point of the error point is taken as the target feature point;

[0129] The average value of the two target feature points is taken as the weld position point.

[0130] The weld position point result obtained by the line segment fitting in the embodiment is shown in the following formula: Figure 7

[0131] In summary, the simulation laser stripe information given by the structured light vision sensor is used, accurate weld positioning is realized, the anti-interference of the identification process is greatly improved, the adaptability is relatively strong, and the application working range of the structured light vision sensor is expanded.

[0132] It should be noted that, for each of the foregoing method embodiments, in order to facilitate description, it is expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other order or simultaneously.

[0133] ​​Based on the same idea as the above-mentioned simulation structured light image-based weld seam recognition method in the embodiments, the application further provides a simulation structured light image-based weld seam recognition system, which can be used to execute the above-mentioned simulation structured light image-based weld seam recognition method. For the convenience of description, only the parts related to the embodiments of the application are shown in the structural schematic diagram of the simulation structured light image-based weld seam recognition system embodiment, and those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and the device can include more or fewer components than the illustrated ones, or combine certain components, or have a different component arrangement.

[0134] As shown in Figure 8 Another embodiment of the application provides a simulation structured light image-based weld seam recognition system, which comprises an image preprocessing module, a candidate point searching module, a candidate point filtering module, a simulation module, an image blocking module, a similarity calculation module and a weld seam position acquisition module.

[0135] The image preprocessing module is used to acquire a weld seam image and perform preprocessing using the Otsu threshold method and morphological closing operation to obtain a preprocessed image.

[0136] The candidate point searching module is used to set an equidistant collection column coordinate group and search for a gray scale vertex of the preprocessed image along the column direction of the gray scale distribution as a laser stripe candidate point.

[0137] The candidate point filtering module is used to calculate the width of the laser stripe candidate point and filter the laser stripe candidate points that are out of the set range.

[0138] The simulation module is used to perform real-time simulation on the weld seam image using a simulation workstation to obtain a simulation image.

[0139] The image blocking module is used to block the preprocessed image to obtain a candidate point block with the filtered laser stripe candidate point as the center and a set width as the edge, and block the simulation image to obtain a simulation block with the filtered laser stripe candidate point as the center and a set width as the edge.

[0140] The similarity calculation module is used to measure the similarity degree of the feature probability distribution of the candidate point block and the corresponding simulation block in the same column in the column coordinate group based on the Bhattacharyya distance, and select the center of the candidate point block with the largest similarity degree in each column as an actual laser stripe point.

[0141] The weld seam position acquisition module is used to perform line segment fitting using the least square method and the random sample consensus method according to the actual laser stripe point, and obtain a weld seam position point according to the weld seam interface type.

[0142] It should be noted that the welding seam identification system based on the simulated structured light image of the present application corresponds to the welding seam identification method based on the simulated structured light image of the present application, and the technical features and advantages described in the above embodiment of the welding seam identification method based on the simulated structured light image are applicable to the embodiment of the welding seam identification system based on the simulated structured light image, and the specific content can be referred to the description in the method embodiment of the present application, which will not be described here again, and hereby declared.

[0143] In addition, in the embodiment of the welding seam identification system based on the simulated structured light image of the above embodiment, the logical division of each program module is only illustrative, and in actual application, the above function distribution can be completed by different program modules according to needs, for example, considering the configuration requirements of the corresponding hardware or the convenience of software implementation, that is, the internal structure of the welding seam identification system based on the simulated structured light image is divided into different program modules to complete all or part of the functions described above.

[0144] As shown in Figure 9 In one embodiment, an electronic device is provided, comprising: at least one processor; and a memory connected in communication with the at least one processor;

[0145] The memory stores computer program instructions (such as a welding seam identification program based on a simulated structured light image) executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the welding seam identification method based on the simulated structured light image described above, specifically:

[0146] Obtain a welding seam image and use Otsu thresholding and morphological closing operation for preprocessing to obtain a preprocessed image;

[0147] Set an equidistantly collected column coordinate group, and search for a gray scale vertex along the column direction of the gray scale distribution of the preprocessed image as a laser stripe candidate point;

[0148] Calculate the width of the laser stripe candidate point, and filter the laser stripe candidate points that exceed the set range;

[0149] Use a simulation workstation to perform real-time simulation on the welding seam image to obtain a simulated image;

[0150] Divide the preprocessed image into blocks with the filtered laser stripe candidate points as the center and a set width as the edge to obtain candidate point blocks; and divide the simulated image into blocks with the filtered laser stripe candidate points as the center and a set width as the edge to obtain simulated blocks;

[0151] Based on the Bath distance, measure the similarity degree of the feature probability distribution of the candidate point blocks and the corresponding simulated blocks in the same column in the column coordinate group, and select the center of the candidate point block with the largest similarity degree in each column as the actual laser stripe point.

[0152] According to the actual laser stripe points, line segment fitting is performed by using a least square method and a random sample consensus method, and the weld position points are obtained according to the weld joint type.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a non-volatile computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiments of each method. Any reference to memory, storage, database or other medium used in each embodiment provided in the present application can include non-volatile and / or volatile memory.

[0154] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of each technical feature in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0155] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any change, modification, substitution, combination, simplification made without departing from the spirit and principle of the present application should be considered as equivalent replacement, and should be included in the protection scope of the present application.

Claims

1. A weld seam recognition method based on simulated structured light images, characterized in that, Includes the following steps: Weld seam images were acquired and preprocessed using the Otsu thresholding method and morphological closing operation to obtain preprocessed images. Set up an equally spaced column coordinate group and search for gray-level vertices in the preprocessed image along the column direction of its gray-level distribution as candidate points for laser stripes; Calculate the width of the candidate laser stripe points and filter out candidate laser stripe points that exceed the set range, specifically as follows: Obtain the width of each laser stripe candidate point W ( i The formula is: W ( i ) = n l + n r ， in, n l for The number of pixels within the window whose gradient is positive or zero; n r for The number of pixels within the window whose gradient is negative or zero; i To set the width w The center pixel of the window range; Determine the width of each candidate point of the laser stripe W ( i Does it belong to [ W min , W max If a point does not belong to the list, remove that candidate point from the laser stripe list. W min and W max The calculation formula is: W min = μ - 3p, W max = μ + 3 σ , ， ， in, μ The average width of all candidate laser stripe points. σ Let the standard deviation of the width of all candidate laser stripe points be . n The number of candidate points for the laser stripe; The weld seam image is simulated in real time using a simulation workstation to obtain the simulation image; Candidate point blocks are obtained by dividing the preprocessed image into blocks centered on the filtered laser stripe candidate points and with a set width as the edge; at the same time, simulation blocks are obtained by dividing the simulation image into blocks centered on the filtered laser stripe candidate points and with a set width as the edge. Based on the Bach distance, the similarity of the feature probability distributions of candidate point blocks and corresponding simulated blocks in the same column of the column coordinate group is measured. The center of the candidate point block with the highest similarity in each column is selected as the actual laser stripe point. Specifically: Calculate the feature probability distribution of the candidate point blocks and the corresponding simulation blocks respectively; Based on the feature probability distribution, the similarity is obtained by calculating the Bach distance between the candidate point block and the corresponding simulation block; The center of the candidate point block with the highest similarity is selected as the actual laser stripe point; Based on the actual laser stripe points, line segments are fitted using the least squares method and random sampling consistency method, and the weld location points are obtained according to the weld interface type.

2. The weld seam recognition method based on simulated structured light images according to claim 1, characterized in that, The equally spaced acquisition column coordinate group is represented as follows: c j = L * j ( j =1,2,…, k )， in, L The set sampling distance, j For collection sequence number, k This represents the maximum number of data collections. Let the grayscale distribution of the column coordinate group be collected at equal intervals. f ( i ), i These are row coordinates, along the grayscale distribution. f ( i A grayscale vertex search is performed on the preprocessed image along the column direction. Pixels that meet the following conditions are selected as candidate points for laser stripes: a: in pixels i Centered on, set width as w Within the window range, f ( i ) is a maximum value, and f ( i (greater than the threshold) T ; b: In The number of pixels with a positive or zero gradient within the window. n l : ; c: In The number of pixels with negative or zero gradients within the window. n r : ; in, η ∈(0,1) is the proportionality coefficient.

3. The weld seam recognition method based on simulated structured light images according to claim 1, characterized in that, The formula for calculating the characteristic probability distribution is: ， ， in, , These are the probability density functions for the simulation block and the corresponding candidate point block, respectively. , These represent the simulation blocks and the corresponding candidate point blocks in terms of features, respectively. u The probability characteristics at each level under the histogram, δ For the Kronecker function, b This refers to the grouping index of pixels in the histogram within the block. t Features u The number of groups in the histogram. x i For the simulation block, the first i The coordinates of each pixel x To simulate the center position of the block, n The number of pixels in the candidate point block is then... The first block of candidate points i The coordinates of each pixel C The normalization coefficients for dividing candidate points into blocks. k The kernel function is the probability density calculation method based on kernel density estimation. n h To simulate the number of pixels in a block, C h The normalization coefficients for the simulation blocks; The formula for calculating the Bartholomew's distance is: ， ， in, B ( p , q The distance between the characteristic probability distributions of candidate point blocks and corresponding simulation blocks in the same column of the column coordinate group is the Bach distance. u As a feature, n u For the number of features, Feature probability distribution for dividing candidate points into blocks This represents the characteristic probability distribution of the corresponding simulation blocks.

4. The weld seam recognition method based on simulated structured light images according to claim 1, characterized in that, The method of fitting line segments using the least squares method and the random sampling consistency method is as follows: Two actual laser stripe points A and B are randomly selected and connected to form a straight line model AB. ε is defined as the threshold that conforms to the straight line model. Calculate the distance from all other actual laser stripe points to the straight line model AB. Actual laser stripe points with a distance less than or equal to ε are designated as interior points, and those with a distance greater than ε are designated as exterior points. After all actual laser stripe points have been calculated, the ratio η of the number of interior points to the total number of actual laser stripe points is calculated. The calculation is completed when η is less than the set ratio threshold η0. The accurate parameters of the straight line model AB are obtained by fitting all interior points using the least squares method. If η is greater than the set ratio threshold η0, then two actual laser stripe points are selected again and connected as a new straight line model, and the above operation is repeated.

5. The weld seam recognition method based on simulated structured light images according to claim 1, characterized in that, The weld interface types include connection type and non-connection type; When the weld interface type of the weld image is connection type, the weld location points are obtained using the linear fitting intersection method, specifically: Obtain the coordinates (x0, y0) of the point closest to the simulated weld from the simulation image; Obtain the two laser-fitted line segments closest to the simulated weld point from the line segment fitting results, denoted as y1=k1x+b1 and y2=k2x+b2 respectively. Solve for the intersection of the two laser-fitted line segments as the weld location point. When the weld interface type of the weld image is non-connected, feature points are extracted first, and then the weld location points are calculated. Specifically: Obtain the coordinates (x1, y1) and (x2, y2) of the leftmost and rightmost actual laser stripe points as search points; Calculate the gradient direction at the search point Then, rotate 90 degrees according to the gradient direction to obtain the search direction. ; Proceed a distance L along the search direction to obtain a test point. If the angle between the gradient direction within 8 adjacent pixels of the test point and the gradient direction of the search point is less than the threshold σ, then the test point is used as the search point to continue the search. Otherwise, it is an error point, the search is stopped, and the search point before the error point is taken as the target feature point. The average value of the two target feature points is taken as the weld location point.

6. A weld seam recognition system based on simulated structured light images, characterized in that, The weld seam recognition method based on simulated structured light images, applied to any one of claims 1-5, includes an image preprocessing module, a candidate point search module, a candidate point filtering module, a simulation module, an image segmentation module, a similarity calculation module, and a weld seam location acquisition module; The image preprocessing module is used to acquire weld seam images and preprocess them using the Otsu thresholding method and morphological closing operation to obtain preprocessed images. The candidate point search module is used to set an equally spaced column coordinate group and search for gray-level vertices in the preprocessed image along the column direction of its gray-level distribution as candidate points for laser stripes. The candidate point filtering module is used to calculate the width of the laser stripe candidate points and filter out laser stripe candidate points that exceed the set range. The simulation module is used to perform real-time simulation of weld images using a simulation workstation to obtain simulated images; The image segmentation module is used to segment the preprocessed image into candidate point blocks with the filtered laser stripe candidate points as the center and the set width as the side; at the same time, it is used to segment the simulation image into simulation blocks with the filtered laser stripe candidate points as the center and the set width as the side. The similarity calculation module is used to measure the similarity of the feature probability distributions of candidate point blocks and corresponding simulated blocks in the same column of the column coordinate group based on the Bach distance, and selects the center of the candidate point block with the highest similarity in each column as the actual laser stripe point. The weld location acquisition module is used to perform line segment fitting based on the actual laser stripe points using the least squares method and random sampling consistency method, and obtain the weld location points according to the weld interface type.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, which enables the at least one processor to perform the weld seam recognition method based on simulated structured light images as described in any one of claims 1-5.

8. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the weld seam recognition method based on simulated structured light images as described in any one of claims 1-5.

Images