Metal structure weld seam recognition method, system and device based on machine vision

Abstract

This invention discloses a machine vision-based method, system, and device for weld seam recognition in metal structural components, relating to the field of welding technology. The method includes extracting a region of interest (ROI) from a 3D model of the metal structural component; the ROI is defined as lines intersecting surfaces or joint lines between components in the 3D model; capturing images of the ROI on the metal structural component; extracting straight lines from the captured images to obtain a set of straight lines; selecting lines that meet positional conditions based on the position of each line in the set to form a weld seam set; converting the image coordinates of each weld seam in the weld seam set to the world coordinates of the 3D model; and transmitting the coordinate-transformed weld seams to the 3D model. This invention, by enabling weld seam recognition and localization, lays the foundation for automated welding of large metal structural components.

Description

Technical Field

[0001] This invention relates to the field of welding technology, and in particular to a method, system and equipment for identifying weld seams in metal structural components based on machine vision. Background Technology

[0002] With the rapid development and continuous progress of China's manufacturing industry, welding technology plays an increasingly important role in industrial production. Moreover, the quality of welding technology directly affects the quality of products. Welding plays a very important role in the production and processing of large steel gates. Many small and medium-sized gate manufacturers produce a small number of products with a wide variety of varieties. They often use manual welding in the production process. This over-reliance on skilled personnel leads to increased production costs and low production efficiency. At the same time, welding produces toxic gases and heat radiation, which can cause certain harm to the human body.

[0003] Therefore, the development of automatic welding equipment is very important. One of the key aspects of automated welding is obtaining the position of the area to be welded. Traditional industrial robots generally use a teaching method for welding. However, small and medium-sized enterprises produce a wide variety of steel gates with inconsistent shapes, and this method requires a lot of time for teaching, resulting in low efficiency.

[0004] For large metal structural components produced in small batches, dedicated positioning fixtures are generally not used to ensure the relative positioning accuracy between the welded sub-components. Instead, the sub-components are often placed manually and then fixed in position using spot welding (tack welding). This results in significant relative positioning errors in large welded components, exceeding the weld position deviations allowed by typical automatic welding machines. Consequently, automatic welding programs based on 3D models cannot function properly. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and device for identifying weld seams in metal structural components based on machine vision.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A machine vision-based method for weld seam recognition in metal structural components includes:

[0008] Extract the Region of Interest (ROI) from the 3D model of the metal structural component; the ROI is the region that includes the intersection lines between faces or the joint lines between sub-components in the 3D model.

[0009] Image capture is performed on the ROI region of the metal structural component;

[0010] Lines are extracted from the captured image to obtain a set of lines;

[0011] Based on the position of each line in the set of lines, lines that meet the position conditions are selected to form a set of welds.

[0012] The image coordinates of each weld in the weld set are transformed into the world coordinates of the 3D model, and the welds after coordinate transformation are transmitted to the 3D model.

[0013] Optionally, the Region of Interest (ROI) can be extracted from the 3D model of the metal structural component, specifically including:

[0014] Obtain the STEP / STP file of the 3D model;

[0015] A STEP / STP file reader is used to read points, lines, and surfaces from the STEP / STP file and extract the Regions of Interest (ROIs) from the 3D model.

[0016] Optionally, image capture is performed on the ROI region on the metal structural component, specifically including:

[0017] A robotic arm is used to move the area scan camera to the corresponding position of the ROI region on the metal structural component;

[0018] The area scan camera is used to capture images of the ROI region on the metal structure.

[0019] Optionally, straight lines are extracted from the captured image to obtain a set of straight lines, specifically including:

[0020] The captured image is sequentially subjected to brightness correction, median filtering, and morphological processing to obtain the preprocessed image;

[0021] The Canny algorithm is used to perform contour detection on the preprocessed image;

[0022] The Hough line search algorithm is used to search for lines in the image after contour detection. The found lines and their distances from the area scan camera are stored in the line set. Lines located on the top, side or bottom of large metal structures are marked according to the ROI region.

[0023] Optionally, before obtaining the preprocessed image, the captured image is subjected to brightness correction, median filtering, and morphological processing in sequence, and the following steps are also included:

[0024] The camera coordinate system of the captured images was determined using the Zhang Zhengyou calibration method.

[0025] Optionally, the positional conditions include a straight line located at the bottom of the metal structure, a straight line being the intersection of two sides, and a straight line located at the top of the metal structure.

[0026] This invention also discloses a machine vision-based weld seam recognition system for metal structural components, comprising:

[0027] The ROI region extraction module is used to extract ROI regions from the 3D model of a metal structural component; the ROI region is the area that includes the intersection lines between faces or the joint lines between sub-components in the 3D model.

[0028] The image capture module is used to capture images of the ROI region on the metal structure using an area scan camera;

[0029] The line extraction module is used to extract lines from the captured image to obtain a set of lines;

[0030] The weld selection module is used to select lines that meet the position conditions in the line set to form a weld set.

[0031] The coordinate transformation module is used to transform the image coordinates of each weld in the weld set to the world coordinates of the three-dimensional model, and then transmit the coordinate-transformed weld to the three-dimensional model.

[0032] The present invention also discloses an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the machine vision-based weld seam recognition method for metal structures.

[0033] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0034] This invention extracts Regions of Interest (ROIs) from a 3D model, then captures images of the ROIs, extracts straight lines from the captured images, and finally identifies welds based on the positions of the extracted lines. The method is simple and easy to implement, expanding the applicability of weld identification. This invention enables the location and identification of welds within weld area images, thereby obtaining weld position information and laying the foundation for automated welding of large metal structural components. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A schematic flowchart of a machine vision-based weld seam recognition method for metal structural components is provided for an embodiment of the present invention.

[0037] Figure 2 A file parsing flowchart provided for embodiments of the present invention;

[0038] Figure 3 The ROI region derived from three-dimensional model analysis is provided in the embodiments of the present invention;

[0039] Figure 4 This is a flowchart of the image capture process provided in an embodiment of the present invention;

[0040] Figure 5 This is an image processing flowchart provided in an embodiment of the present invention;

[0041] Figure 6 A flowchart for weld seam identification provided in an embodiment of the present invention;

[0042] Figure 7 This is a flowchart of weld result processing provided in an embodiment of the present invention;

[0043] Figure 8 This invention provides a schematic diagram of a machine vision-based weld seam recognition system for metal structural components. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] The purpose of this invention is to provide a method, system, and device for identifying weld seams in metal structural components based on machine vision. This invention enables the location and identification of weld seams in weld seam area images, thereby obtaining the location information of the weld seams and laying the foundation for automated welding of large metal structural components.

[0046] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Example 1

[0048] like Figure 1 As shown, this embodiment provides a machine vision-based method for weld seam recognition in metal structural components, including:

[0049] Step 101: Extract the Region of Interest (ROI) from the 3D model of the metal structure; the ROI is the region that contains the intersection lines between faces or the joint lines between sub-components in the 3D model.

[0050] Step 102: Capture the image of the ROI region on the metal structural component.

[0051] Step 103: Extract lines from the captured image to obtain a set of lines.

[0052] Step 104: Based on the position of each line in the line set, select the lines that meet the position conditions to form the weld set.

[0053] Step 105: Convert the image coordinates of each weld in the weld set to the world coordinates of the 3D model, and transfer the coordinate-transformed weld to the 3D model.

[0054] This embodiment is based on obtaining a rough 3D model of a metal structural component through rapid 3D reconstruction. The metal structural component is a large non-standard structural component built with T-shaped components. The structural information of the rapidly reconstructed 3D model is rough and inaccurate. Since the weld feature information of this 3D model is fuzzy, in order to obtain specific weld information, it is necessary to further process the image information based on this 3D model and in conjunction with an area array camera.

[0055] Step 101 specifically includes:

[0056] Creating a STEP / STP (STEP or STP) file reader involves: importing the corresponding API library, setting up the environment variables required by the reader, creating the file reader, and reading all points, lines, and surfaces of the 3D model.

[0057] Obtaining the STEP / STP file of the 3D model specifically includes: selecting a STEP / STP graphic file for rapid reconstruction locally, reading the STEP / STP file based on the SolidWorks open-source code library, and reading the shape features of the 3D model into local memory.

[0058] A STEP / STP file reader is used to read points, lines, and surfaces from the STEP / STP file and extract the region of interest (ROI) from the 3D model.

[0059] Coordinate transformation is performed on the ROI region to convert the world coordinates of the 3D model to the robot's coordinate system, which facilitates subsequent image capture.

[0060] Step 102 specifically includes:

[0061] The robotic arm is used to move the area scan camera to the corresponding position of the ROI region on the metal structure. Specifically, this includes setting the joint parameters of the robotic arm and moving the area scan camera to the ROI region obtained by analyzing the 3D model.

[0062] The area scan camera is used to capture images of the ROI region on the metal structure. Specifically, the area scan camera uses a soft-trigger method to capture images of the target area and saves the captured images to local memory for subsequent image processing.

[0063] Step 103 specifically includes:

[0064] The camera coordinate system of the captured images is determined using the Zhang Zhengyou calibration method to facilitate the conversion of subsequent image processing results.

[0065] Read the images captured by the area scan camera from memory.

[0066] The captured image is sequentially subjected to brightness correction, median filtering, and morphological processing to eliminate noise in the image and ensure that as many straight lines as possible are identified in the image, resulting in a preprocessed image.

[0067] The Canny algorithm is used to perform contour detection on the preprocessed image to obtain approximate contour information, which facilitates the operation of the subsequent weld seam recognition and positioning module.

[0068] Feature processing is performed on the image after contour detection. The Hough line search algorithm is used to search for lines in the image after contour detection. The found lines and the distances between each found line and the area scan camera are stored in the line set. The ROI region is sequentially marked to indicate that the lines in the line set are located at the top, side or bottom of the large metal structure. Specifically, this includes: obtaining the results of the laser displacement sensor at the end of the area scan camera, merging the endpoint and length information of line li, the corresponding distance data of the laser displacement sensor and the marked position information into the information of line li, and loading it into the line set L.

[0069] In step 104, based on the special structural characteristics of the metal structural components, the straight lines that meet the position conditions are selected to form a weld set according to the position of each straight line in the set of straight lines.

[0070] The positional conditions include a straight line located at the bottom of the metal structure, a straight line being the intersection of two sides, and a straight line located at the top of the metal structure.

[0071] Step 104 specifically includes:

[0072] Step 1041: Read a line l from the set of lines L{l1,l2,l3,…} i ∈L.

[0073] Step 1042, find the straight line l i The position of the line can be determined by the location information marked in the image.

[0074] Step 1043, if the read line l i Located at the bottom of a large metal structural component, at the intersection of the web and the base plate, this straight line is a fillet weld, proceed to step 1046.

[0075] Step 1044: If the read line l i If the straight line is located on the side of a large metal structural component and is the intersection of two webs, then this straight line is a vertical weld, and step 1046 is executed.

[0076] Step 1045, if the read line l i If the weld is located at the top of a large metal structural component and at the junction of the two flanges, then the straight line is a flat weld; otherwise, proceed to step 1047.

[0077] Step 1046: Load the straight line information into the weld set H.

[0078] Step 1047: Determine whether the set of lines L has been read completely. If not, repeat steps 1041 to 1047.

[0079] In step 105, such as Figure 7 As shown, the weld information is classified and partitioned, and the processing results are transformed into coordinates and transmitted to the 3D model to improve the weld information on the 3D model for subsequent automated welding. Specifically, this includes:

[0080] Step 1051, based on line l i The location information marked in the image classifies lines located in the same area into the same category.

[0081] The region in step 1051 includes the bottom, sides, and top, according to line l i The three-dimensional information is used to partition the line.

[0082] Step 1052: Perform coordinate transformation on all welds in sequence, converting the image coordinates of the welds to the world coordinates of the 3D model.

[0083] Step 1053: Transmit the weld information in the world coordinate system to the 3D model, and call the 3D model processing module to improve the weld information on the 3D model to facilitate subsequent welding.

[0084] The machine vision-based weld seam recognition method for metal structural components of this invention, after executing all the above steps, performs feature analysis on the approximate 3D model of the rapidly reconstructed large metal structural component, and uses an area array camera to locate and identify the weld seam information in the ROI region, obtaining the position information of the area to be welded on the large metal structural component, namely the coordinates and length information of the weld seam. This method can identify and locate the area to be welded on large metal structural components without the need for specific model information of the structural component. However, small and medium-sized gate processing enterprises often process small batches of diverse products, and often lack dedicated positioning fixtures. The error between the structure after manual positioning and fixing by workers and the 3D model exceeds the allowable deviation of the weld seam of the automatic welding robot, making it impossible to program the welding work based on the 3D model. Using conventional teaching for welding is time-consuming, labor-intensive, and inefficient. The method of this invention can adapt to different types of large metal structural parts and calculate accurate weld position information. It solves the problem of low efficiency caused by conventional teaching when frequently changing welding objects. The overall design has the characteristics of wide applicability, simplicity and ease of implementation, and high degree of automation. Moreover, this algorithm is developed based on the .NET platform, and the source code of the algorithm used is completely free. The structure of this invention is simple, easy to operate, and the recognition rate can reach more than 85%.

[0085] Example 2

[0086] This embodiment provides a machine vision-based method for identifying weld seams in metal structural components, including the following steps:

[0087] (1) File parsing: The program parses and reads the STEP / STP file of the 3D model obtained by rapid reconstruction, performs structural analysis on the 3D model, and finds ROI areas where welds may exist.

[0088] like Figure 2 As shown, the parsing process after reading the STEP / STP file is as follows:

[0089] (11) Create a STEP / STP file reader.

[0090] (12) Introduce the corresponding API class library, set up the environment variables required by the reader, create a file reader, and read all points, lines and surfaces of the 3D model.

[0091] (13) Select the STEP / STP graphic file of the rough 3D model that is quickly reconstructed locally, and read the STEP / STP file based on the SolidWorks open source code library.

[0092] (14) Read the shape features of the model into memory, analyze the model to obtain the ROI region, such as Figure 3 As shown.

[0093] (15) Perform coordinate transformation on the ROI region to transform the world coordinates of the 3D model into the robot's coordinate system, which facilitates the subsequent image capture.

[0094] (2) Image capture: The area array camera is moved by a robotic arm to capture images of the ROI region.

[0095] like Figure 4 As shown, the steps for image capture are as follows:

[0096] (21) Set the joint parameters of the robot arm and move the area array camera to the ROI area obtained by analyzing the rough 3D model reconstructed quickly.

[0097] (22) The camera uses a soft-trigger method to capture images of the target area.

[0098] (23) Save the captured image to local memory for subsequent image processing.

[0099] (3) Image processing: Use image processing tools to analyze the captured image and obtain the position information of all lines in the ROI region.

[0100] like Figure 5 As shown, the steps of image processing are as follows:

[0101] (31) The camera is calibrated using the Zhang Zhengyou calibration method to determine the camera coordinate system, which facilitates the conversion of subsequent image processing results. The intrinsic and extrinsic parameter matrices of the camera can be obtained based on the Zhang Zhengyou calibration method, and the four coordinate systems are connected.

[0102] When the coordinates are transformed from the image coordinate system UO0V to the imaging plane coordinate system O P X P Y P Z P Then, the relationship between pixels and physical units was established. Let the pixel coordinates of a point on the image be (c x c y If the point (u, v) on the image coordinate system is different from the point (x, v) on the imaging plane, then the coordinates of the point (u, v) on the image coordinate system are different from the coordinates of the point (x, v) on the imaging plane. p y p The relationship is as follows:

[0103]

[0104] Among them, s x s y These represent the number of pixels per millimeter in the x and y directions of the imager, respectively.

[0105] When the camera coordinate system O c X c Y c Z cWith the imaging plane coordinate system O P X P Y P Z P After the transformation, the distance between the coordinates matches the real-world distance, so the point (x) on the imaging plane... p y p (x, 0) and the point (x) in the camera coordinate system c y c , z c The relationship is as follows:

[0106]

[0107] Where f represents the focal length of the camera (area scan camera).

[0108] Final world coordinate system (x w y w , z w ) and camera coordinate system (x) c y c , z c The conversion relationship is as follows:

[0109]

[0110] In this context, R represents the rotation matrix in the camera extrinsic matrix, and T represents the translation matrix in the camera extrinsic matrix.

[0111] Combining the above equations, we can obtain the transformation matrix from pixel coordinates to world coordinates as follows:

[0112]

[0113] Z C The distance between the camera and the target object is represented by A, which needs to be measured on-site. A represents the camera's intrinsic parameter matrix.

[0114] (32) Read the target image captured by the area array camera from memory.

[0115] (33) The captured image is subjected to brightness correction, median filtering, morphological processing and other operations to eliminate noise in the image and ensure that as many straight lines as possible are identified in the image.

[0116] (34) Use the Canny algorithm to perform contour detection on the preprocessed image to obtain approximate contour information, which facilitates the operation of the subsequent weld seam identification and positioning module.

[0117] (35) Perform feature processing on the image after contour detection, and call the Hough line search algorithm in the weld seam recognition and positioning module to find the line in the image.

[0118] (36) Obtain the results from the laser displacement sensor at the end of the area array camera, and then transfer the line l i The endpoint and length information, the corresponding distance data from the laser displacement sensor, and the marked position information are combined into a straight line l. i The information is loaded into set L.

[0119] (4) Weld identification: According to the actual situation: After image processing, a set of straight lines will be obtained. Some of the lines in the set do not belong to the weld. Therefore, in order to improve the accuracy of the weld location, the set of straight lines is filtered. Due to the special nature of large metal structural parts, the area where the weld exists is relatively fixed. Based on the information of the straight lines in the set of straight lines, the straight lines that belong to the weld are filtered and loaded into the weld set H.

[0120] like Figure 6 As shown, the specific steps for weld identification are as follows:

[0121] (41) Read a line l from the set of lines L{l1,l2,l3,…} i ∈L.

[0122] (42) Find the straight line l i The position of the line can be determined by the location information marked in the image.

[0123] (43) If the line l is read i Located at the bottom of the large metal structural component, at the intersection of the web and the base plate, this straight line is the fillet weld, proceed to step S46.

[0124] (44) If the line l is read i If the straight line is located on the side of a large metal structural component and is the intersection of two webs, then this straight line is a vertical weld, and step S46 is executed.

[0125] (45) If the line l is read i If the line is located at the top of a large metal structural component and at the junction of the two flanges, then this line is a flat weld; otherwise, proceed to step S47.

[0126] (46) Load the straight line information into the weld line set H.

[0127] (47) Determine whether the set of lines L has been read. If not, repeat steps S41 to S47.

[0128] (5) Weld result processing: The weld information is classified and partitioned, and the processing results are transformed into coordinates. The results are then transmitted to the three-dimensional model to improve the weld information on the three-dimensional model so as to facilitate the subsequent realization of automated welding.

[0129] The specific steps for processing weld results are as follows:

[0130] (51) Based on line l i The marked location information categorizes straight lines within the same area into the same type.

[0131] (52) Perform coordinate transformation on all welds in sequence to transform the image coordinates of the welds into the world coordinates of the three-dimensional model.

[0132] Based on the transformation matrix from pixel coordinates to world coordinates obtained from camera calibration:

[0133]

[0134] Inverse solution for world coordinates (x) w ,y w ,z w The above formula can be transformed into the following steps:

[0135]

[0136] Set intermediate parameters Mat2 = R -1 T, observe only the third term on both sides of the above equation:

[0137] z w =Z c *Mat1(2,0)-Mat2(2,0);

[0138] Z c =(z w +Mat2(2,0)) / Mat1(2,0);

[0139] Mat2(2,0) represents the third term of the 3×1 matrix. The distance dis from the camera to the target object can be obtained from the laser displacement sensor, and the height of the camera is h. c Therefore, the world coordinates z of the measured target point can be determined. w =h c -dis, substituting it into the formula, can calculate Z. c World coordinates can be obtained from pixel coordinates.

[0140] (53) Transmit the information of the weld in the world coordinate system to the rough three-dimensional model, and call the three-dimensional model processing module to improve the information of the weld on the three-dimensional model so as to facilitate subsequent welding.

[0141] In summary, after all five steps have been performed, the weld information of the ROI region can be identified and processed using an area array camera based on the features of the approximate 3D model obtained by rapidly reconstructing the large metal structure. This process improves the position information of the area to be welded on the 3D model of the large metal structure, thus obtaining the coordinates and length information of the weld.

[0142] For practical use cases, the first step is to read the rapidly reconstructed STEP / STP graphic file of the 3D model locally using the SolidWorks open-source code library. Simultaneously, the structural information of the 3D model is analyzed to determine the location information of potential Regions of Interest (ROIs) where welds may exist. This provides location information for subsequent image capture of the ROIs. Figure 3 As shown, the intersection lines between the base plate 1 and the web plate 2, and between the web plate 2 and the flange plate 3, represent the potential Regions of Interest (ROIs) for the weld. By using a robotic arm to move the area array camera to the target location for image capture, and then processing the captured weld images using image processing tools, the specific information of all straight lines within the ROI is obtained. The positions of the straight lines are then determined, and the weld line information is filtered out. The weld line information is then transformed from pixel coordinates to the world coordinate system of the 3D model. The transformed weld information is then transmitted to the 3D model module, thereby refining the accurate weld information on the approximate 3D model.

[0143] Example 3

[0144] like Figure 8 As shown, this embodiment provides a machine vision-based weld seam recognition system for metal structural components, including:

[0145] The ROI region extraction module 201 is used to extract ROI regions from the three-dimensional model of the metal structure; the ROI region is the line where the surfaces intersect in the three-dimensional model.

[0146] Image capture module 202 is used to capture images of the ROI region on the metal structure using an area scan camera.

[0147] The line extraction module 203 is used to extract lines from the captured image to obtain a set of lines.

[0148] The weld selection module 204 is used to select lines that meet the position conditions based on the position of each line in the line set to form a weld set.

[0149] The coordinate transformation module 205 is used to transform the image coordinates of each weld in the weld set to the world coordinates of the three-dimensional model, and to transmit the coordinate-transformed weld to the three-dimensional model.

[0150] Example 3

[0151] This invention provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to cause the electronic device to perform the method of embodiment 1.

[0152] Alternatively, the aforementioned electronic device may be a server.

[0153] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of embodiment 1.

[0154] Any references to memory, storage, database, or other media used in the embodiments provided in this invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0155] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0156] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for identifying a weld of a metal structure based on machine vision, characterized by, include: Extracting the Region of Interest (ROI) from the 3D model of a metal structural component; The ROI region is the area that includes the intersection lines between faces or the joint lines between components in the 3D model. The image capture of the ROI region on the metal structure specifically includes: using a robotic arm to move an area scan camera to the corresponding position of the ROI region on the metal structure; and using the area scan camera to capture the image of the ROI region on the metal structure. Lines are extracted from the captured image to obtain a set of lines; Based on the position of each line in the set of lines, lines that meet the position conditions are selected to form a set of welds; the position conditions include the line being located at the bottom of the metal structure, the line being the intersection of two sides, and the line being located at the top of the metal structure. The image coordinates of each weld in the weld set are converted to the world coordinates of the three-dimensional model, and the weld after coordinate conversion is transmitted to the three-dimensional model to improve the weld information on the three-dimensional model, that is, to obtain the position coordinates and length information of the weld, so as to facilitate the subsequent realization of automated welding and adapt to different types of large metal structural parts. Based on the position of each line in the set of lines, lines that meet the positional conditions are selected to form a set of weld seams, specifically including: Step 1041, reading a straight line from the set of straight lines ;​ Step 1042, finding the straight line the position information marked in the middle to find the position of the straight line; Step 1043, if the read straight line Located at the bottom of the metal structural member, at the intersection of the web and the bottom plate, this straight line is a fillet weld, proceed to step 1046; Step 1044: If the read straight line If the straight line is located on the side of the metal structural member and is the intersection of two webs, then the straight line is a vertical weld, and step 1046 is executed. Step 1045, if the straight line is located at the top of the metal structure and the weld is located at the intersection of the two wing plates, then this straight line is a flat weld, otherwise step 1047 is executed; Step 1046: Load the straight line information into the weld seam set; Step 1047, judging the straight line set whether reading is completed, if not, repeating steps 1041 to 1047; Extracting the Region of Interest (ROI) from the 3D model of a metal structural component, specifically including: Obtain the STEP / STP file of the 3D model; A STEP / STP file reader is used to read points, lines, and surfaces from the STEP / STP file and extract the Regions of Interest (ROIs) into the 3D model.

2. The machine vision-based metal structural member weld joint recognition method according to claim 1, characterized by, Lines are extracted from the captured image to obtain a set of lines, specifically including: The captured image is sequentially subjected to brightness correction, median filtering, and morphological processing to obtain the preprocessed image; The Canny algorithm is used to perform contour detection on the preprocessed image; The Hough line search algorithm is used to search for lines in the image after contour detection. The found lines and their distances from the area scan camera are stored in the line set. The lines located at the top, side or bottom of the metal structure are marked in the line set according to the ROI region.

3. The machine vision-based metal structural member weld joint recognition method according to claim 2, characterized by, Before obtaining the preprocessed image, the captured image undergoes sequential brightness correction, median filtering, and morphological processing. The camera coordinate system of the captured images was determined using the Zhang Zhengyou calibration method.

4. A machine vision-based weld joint identification system for metal structures, characterized by The machine vision-based metal structural weld seam recognition system applies the machine vision-based metal structural weld seam recognition method of claim 1, wherein the machine vision-based metal structural weld seam recognition system comprises: The ROI region extraction module is used to extract ROI regions from the 3D model of a metal structural component; the ROI region is the area that includes the intersection lines between faces or the joint lines between sub-components in the 3D model. The image capture module is used to capture images of the ROI region on the metal structure using an area scan camera; The line extraction module is used to extract lines from the captured image to obtain a set of lines; The weld selection module is used to select lines that meet the position conditions in the line set to form a weld set. The coordinate transformation module is used to transform the image coordinates of each weld in the weld set to the world coordinates of the three-dimensional model, and then transmit the coordinate-transformed weld to the three-dimensional model.

5. An electronic device, comprising: The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the method according to any one of claims 1 to 3.

Images