Weld positioning and dimension calculation method and device, storage medium and electronic equipment

Abstract

The application relates to a weld positioning and size calculation method and device, a storage medium and electronic equipment, wherein the method comprises the following steps: acquiring overall three-dimensional point cloud data of each welded area through a laser projector and a color camera, and acquiring overall color images of each welded area through the color camera; extracting a target image corresponding to a region of interest from the overall color images, the region of interest being a region of interest of a target weld in the corresponding welded area; extracting a positioning area image of the target weld from the target image according to a preset rule; extracting target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data corresponding to the target weld, and performing point cloud principal component analysis on the target three-dimensional point cloud data to obtain the weld size of the target weld. The application has the effect of improving the accuracy of the weld size detection result.

Description

Technical Field

[0001] This application relates to the field of weld size calculation technology, specifically to a weld positioning and size calculation method, device, storage medium, and electronic device. Background Technology

[0002] A weld is a joint formed by melting and connecting the welding rod and the metal at the joint using the high temperature of a welding heat source. After the weld metal cools, the two workpieces are welded together as a whole. Welding is a process technology that joins metals by heating, high temperature, or high pressure. With the widespread use of reinforced steel materials in national infrastructure construction such as roads, buildings, and bridges, the demand for connections between reinforcing bars is increasing. Welding is the primary method of connecting reinforcing bars, and the quality of the weld determines the quality of the reinforcing bar connection, the performance of the resulting reinforced steel frame, and ultimately, the safety and stability of the entire construction project.

[0003] Among these, the weld size of reinforcing bars is an important standard for judging weld quality. Therefore, the calculation of weld size is a crucial step in the weld quality assessment process. Currently, weld size is mainly measured manually by hand, resulting in low accuracy of the measurement results. Summary of the Invention

[0004] To improve the accuracy of weld size detection results, this application provides a weld positioning and size calculation method, apparatus, storage medium, and electronic device.

[0005] The first aspect of this application provides a method for weld seam positioning and dimension calculation, applied to a 3D laser scanner, wherein the 3D laser scanner is equipped with a color camera and a laser projector, specifically including: The laser projector and the color camera are used to acquire overall three-dimensional point cloud data of each welded area, and the color camera is used to acquire overall color images of each welded area. Extract the target image corresponding to the region of interest from the overall color image, wherein the region of interest is the region of interest of the target weld in the corresponding welded area; The positioning area image of the target weld is extracted from the target image according to preset rules; Extract the target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data of the welded area, and perform principal component analysis on the target three-dimensional point cloud data to obtain the weld size of the target weld.

[0006] By employing the above technical solution, after acquiring overall 3D point cloud data and overall color images of multiple welded areas, the region of interest (ROI) of the target weld in each welded area is determined. The target image corresponding to this ROI is then extracted from the overall color image of the welded area, facilitating further analysis of the ROI and aiding in the localization of the target weld. Next, various noise interferences are eliminated, and the localization area image of the target weld is accurately extracted from the target image. Finally, based on the conversion relationship between the color camera and the laser projector, the target 3D point cloud data corresponding to the pixels of the localization area image is extracted from the overall 3D point cloud data corresponding to the welded area. Principal component analysis of the target 3D point cloud data is then performed to ultimately determine the weld size of the target weld, eliminating the need for manual inspection and thus improving the accuracy of weld size detection results.

[0007] Optionally, extracting the target image corresponding to the region of interest from the overall color image specifically includes: Obtain the target minimum field of view corresponding to the target weld, wherein the target minimum field of view is the minimum field of view required for the target weld to be fully scanned by the three-dimensional laser scanner; The maximum and minimum values ​​of the X-axis, Y-axis and Z-axis directions of the minimum field of view of the target are selected respectively to obtain the target extreme coordinates corresponding to the target weld, and the target extreme coordinates cover the target weld; The target image corresponding to the region of interest is obtained by cropping the image within the target extreme value coordinates from the overall color image.

[0008] By employing the above technical solution, the target minimum field of view of the target weld is determined, enabling the 3D laser scanner to completely scan the target weld within this minimum field of view. Then, based on the coordinate magnitude, the maximum and minimum coordinate values ​​in the X, Y, and Z axes of the target minimum field of view are selected respectively. This yields the range encompassed by the target extreme coordinates, which includes the target weld while minimizing the target weld's location area. Finally, an image is cropped from the overall color image according to the range encompassed by the target extreme coordinates, ultimately determining the target image corresponding to the region of interest. This facilitates further precise positioning of the target weld based on the target image.

[0009] Optionally, extracting the positioning area image of the target weld from the target image according to a preset rule specifically includes: The target image is segmented, converted to grayscale, binarized, dilated, and its contour is extracted to obtain the maximum contour corresponding to the target weld. Based on the maximum contour, the positioning area image of the target weld is extracted from the target image.

[0010] By employing the above technical solution, after the target image is determined, image segmentation and binarization are performed on the target image to effectively remove noise in the target weld. Grayscale processing reduces the memory footprint of the target image and improves processing speed. Next, contour extraction is performed to obtain the maximum contour corresponding to the target weld, i.e., the precise contour of the target weld. Finally, based on the determined maximum contour, the positioning region image corresponding to this maximum contour is extracted (cropped) from the target image, thereby achieving accurate positioning of the target weld in the overall color image.

[0011] Optionally, the step of performing image segmentation, grayscale conversion, binarization, dilation processing, and contour extraction on the target image to obtain the maximum contour corresponding to the target weld seam specifically includes: The target image is segmented using a preset image segmentation algorithm to obtain multiple segmented images; Each segmented image is sequentially subjected to grayscale conversion, binarization, dilation processing, and contour extraction to obtain multiple contours; Calculate the perimeter of each of the aforementioned contours, and determine the contour with the largest perimeter as the maximum contour corresponding to the target weld.

[0012] By adopting the above technical solution, the target image is segmented into multiple segmented images using an image segmentation algorithm. Then, each segmented image is sequentially processed by grayscale conversion, binarization, and dilation to obtain multiple processed segmented images. Further, contour extraction is performed on each of the multiple processed segmented images to obtain multiple contours. Finally, contours with smaller perimeters are discarded because they may be image noise. The contour with the largest perimeter is selected as the largest contour of the target weld, thereby obtaining a more accurate contour of the target weld.

[0013] Optionally, the step of extracting the target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data corresponding to the target weld, and performing principal component analysis on the target three-dimensional point cloud data to obtain the weld size of the target weld, specifically includes: Obtain the coordinate transformation matrix between the first coordinate system corresponding to the color camera and the second coordinate system corresponding to the laser projector; According to the coordinate transformation matrix, the positioning area image of the target weld is transformed from the first coordinate system to the second coordinate system to obtain the target position information of the positioning area image. The target position information is the relative positional relationship of the positioning area image to the overall color image. Based on the target location information, extract the target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data corresponding to the target weld; Principal component analysis of the target three-dimensional point cloud data is performed to obtain the weld size of the target weld.

[0014] By adopting the above technical solution, after the positioning area image of the target weld is determined, since the positioning area image is in the first coordinate system and the second coordinate system is in three-dimensional space, the positioning area image is transformed from the first coordinate system to the second coordinate system according to the coordinate transformation matrix between the first and second coordinate systems, thus obtaining the target position information of the positioning area image. Next, based on this target position information, corresponding target three-dimensional point cloud data is selected from the overall three-dimensional point cloud data, that is, the target three-dimensional point cloud data is selected based on the relative positional relationship in three-dimensional space. Finally, principal component analysis is performed on the target three-dimensional point cloud data to ultimately determine the weld size with high accuracy.

[0015] Optionally, the step of performing principal component analysis on the target three-dimensional point cloud data to obtain the weld size of the target weld specifically includes: The target three-dimensional point cloud data is analyzed using a preset principal component analysis method to obtain the weld size of the target weld.

[0016] By adopting the above technical solution, principal component analysis can not only effectively reduce redundant information in the target 3D point cloud data, but also remove noise from the target 3D point cloud data, thereby improving its accuracy and reliability, and ultimately making the calculation of weld size more accurate.

[0017] Optionally, before acquiring the overall three-dimensional point cloud data of each welded area using the laser projector and the color camera, the method further includes: Determine whether each welded area is within the measurement range of the three-dimensional laser scanner; The laser projector and the color camera acquire overall three-dimensional point cloud data of each welded area, and the color camera acquires overall color images of each welded area, specifically including: With each welded area within the measurement range of the 3D laser scanner, the overall 3D point cloud data of each welded area is acquired by the laser projector and the color camera, and the overall color image of each welded area is acquired by the color camera.

[0018] By adopting the above technical solution, since the measurement range of the 3D laser scanner is limited, it is necessary to determine whether each welded area is within the measurement range of the 3D laser scanner before acquiring the overall 3D point cloud data and overall color image of each welded area. This ensures that each welded area can be scanned and measured, thereby enabling the acquisition of overall color image and overall 3D point cloud data of each welded area without omission.

[0019] A second aspect of this application provides a weld positioning and dimension calculation device, specifically comprising: The information acquisition module is used to acquire overall three-dimensional point cloud data of each welded area through a laser projector and a color camera, and to acquire overall color images of each welded area through the color camera. The image extraction module is used to extract the target image corresponding to the region of interest from the overall color image, wherein the region of interest is the region of interest of the target weld in the corresponding welded area; A weld seam positioning module is used to extract the positioning area image of the target weld seam from the target image according to preset rules; The size determination module is used to extract the target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data corresponding to the target weld, and perform point cloud principal component analysis on the target three-dimensional point cloud data to obtain the weld size of the target weld.

[0020] By adopting the above technical solution, the information acquisition module acquires the overall three-dimensional point cloud data of each welded area through the laser projector and color camera inside the three-dimensional laser scanner, and acquires the overall color image of each welded area separately through the color camera. Then, the image extraction module extracts the target image corresponding to the region of interest from the overall color image. Next, the weld positioning module extracts the positioning area image of the target weld from the target image according to preset rules. Finally, the size determination module extracts the target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data, and performs point cloud principal component analysis on the target three-dimensional point cloud data to finally obtain the weld size of the target weld.

[0021] In summary, this application includes at least one of the following beneficial technical effects: After acquiring overall 3D point cloud data and overall color images of multiple welded areas, the region of interest (ROI) for the target weld in each welded area is determined. The corresponding target image is then extracted from the overall color image of the welded area to facilitate further analysis of the ROI and aid in the localization of the target weld. Next, various noise interferences are eliminated, and the localization region image of the target weld is accurately extracted from the target image. Finally, based on the conversion relationship between the color camera and the laser projector, the target 3D point cloud data corresponding to the pixels of the localization region image is extracted from the overall 3D point cloud data of the welded area. Principal component analysis is performed on the target 3D point cloud data to ultimately determine the weld size of the target weld, eliminating the need for manual inspection and thus improving the accuracy of weld size detection results. Attached Figure Description

[0022] Figure 1This is a flowchart illustrating a weld positioning and dimension calculation method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the overall three-dimensional point cloud data and overall color image of a welded area provided in an embodiment of this application; Figure 3 This is a schematic diagram of a region of interest for a target weld provided in an embodiment of this application; Figure 4 This is a flowchart of a weld size calculation method provided in an embodiment of this application; Figure 5 This is a flowchart illustrating another method for weld positioning and dimension calculation provided in an embodiment of this application; Figure 6 This is a schematic flowchart illustrating the determination of the maximum profile of a target weld seam according to an embodiment of this application. Figure 7 This is a schematic diagram of the structure of a weld positioning and dimension calculation device provided in an embodiment of this application; Figure 8 This is a schematic diagram of another weld positioning and dimension calculation device provided in the embodiments of this application.

[0023] Explanation of reference numerals in the attached drawings: 11. Information acquisition module; 12. Image extraction module; 13. Weld positioning module; 14. Dimension determination module. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0025] In the description of the embodiments in this application, words such as "illustrative," "for example," or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "illustrative," "for example," or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "illustrative," "for example," or "for example" is intended to present the relevant concepts in a specific manner.

[0026] In the description of the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0027] A three-dimensional laser scanner primarily utilizes the principle of laser ranging. It collects information such as the three-dimensional coordinates, texture, and reflectivity of numerous points on the surface of the object being measured (in this embodiment, the weld surface), reconstructs its lines, surfaces, volumes, and three-dimensional model data, and ultimately determines the three-dimensional point cloud data of the weld surface. In this embodiment, the three-dimensional laser scanner is equipped with a laser projector and a color camera. It should be noted that the weld positioning and dimension calculation method disclosed in this application is applied to the three-dimensional laser scanner; that is, the executing entity is the three-dimensional laser scanner.

[0028] See Figure 1 This application discloses a flowchart illustrating a method for weld positioning and dimension calculation, which can be implemented using a computer program or run on a weld positioning and dimension calculation device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application, specifically including: S101: Acquire overall 3D point cloud data of each welded area using a laser projector and a color camera, and acquire overall color images of each welded area using a color camera.

[0029] Specifically, a laser projector is a projector that uses a laser beam to project an image. A color camera, also known as an industrial camera in a 3D laser scanner, essentially converts light signals into ordered electrical signals. Color cameras typically use optical scanning, employing an internal optical system to scan objects and then using photoelectric conversion and signal processing technologies to convert the scanned image into a digital signal, which is then processed and analyzed by a computer.

[0030] 3D point cloud data is a dataset composed of the coordinates and color information of points in three-dimensional space. It can be used in many fields, such as 3D printing, industrial inspection, medical imaging, and geophysics. It is typically acquired by devices from various fields, such as 3D laser scanners, cameras, and radar. These devices scan objects in three-dimensional space and record the coordinates and color information of points on the object's surface. Point cloud data can be processed and analyzed in various ways, such as feature extraction, target identification, and model analysis.

[0031] Each welded area consists of a single welded rebar and its corresponding weld at a single welding station. In other embodiments, a welded area may contain a single welded rebar.

[0032] In this embodiment, a laser projector first projects a stripe image onto the welded area. Then, a color camera captures modulated stripe images of the surface of each welded area. Finally, the captured images are processed using a preset structured light algorithm to obtain the overall 3D point cloud data of each welded area. Alternatively, the color camera inside a 3D laser scanner can directly capture images of each welded area to obtain an overall color image of each welded area. See details... Figure 2 Structured light algorithm is a 3D depth estimation algorithm based on the geometric relationship of similar triangles. This algorithm directly marks the surface of an object in 3D space by projecting highly pseudo-random laser speckle, and then calculates the object's depth information based on the laser speckle. Structured light algorithm can be applied to many fields in 3D vision, such as computer vision, robot navigation, and medical image diagnosis. This is existing technology and will not be elaborated further.

[0033] S102: Extract the target image corresponding to the region of interest from the overall color image. The region of interest is the region of interest of the target weld in the corresponding welded area.

[0034] In one feasible implementation, the target minimum field of view corresponding to the target weld is obtained, which is the minimum field of view required for the target weld to be fully scanned by a 3D laser scanner. The maximum and minimum values ​​of the X-axis, Y-axis and Z-axis directions of the minimum field of view of the target are selected respectively to obtain the target extreme value coordinates corresponding to the target weld. The target extreme value coordinates cover the target weld. The target image corresponding to the region of interest is obtained by cropping the image within the extreme coordinates of the target from the overall color image.

[0035] Specifically, after the overall 3D point cloud data and overall color image are determined, for each welded area, the corresponding overall 3D point cloud data and overall color image involve not only the target weld, but also the welded workpiece (reinforcing bar) and welding station, etc. It is necessary to gradually extract color images containing only the target weld from the overall color image. Therefore, the target image corresponding to the region of interest (ROI) of the target weld is extracted from the overall color image. In the field of image processing, the region of interest (ROI) is a selected image area delineated by a rectangle, circle, ellipse, irregular polygon, etc. This image area is the focus of image analysis, facilitating further identification of the target weld. This reduces the amount of image data processing, speeding up the process, and avoids the influence of noise interference from other areas. In this embodiment, a rectangle is used to extract the ROI; in other embodiments, a circle can also be used.

[0036] The field of view is the maximum range that a camera can observe, usually expressed in angles. The larger the field of view, the larger the observation range. For this application, the field of view is the maximum range that a 3D laser scanner can observe. The target minimum field of view is the minimum field of view required for each target weld to be completely scanned by the 3D laser scanner. Specifically, it is obtained by: ensuring that the target weld is completely scanned, adjusting the field of view size to the minimum in advance to obtain the target minimum field of view, and recording the target minimum field of view corresponding to the target weld.

[0037] Another feasible method for extracting the target image corresponding to the region of interest is as follows: After determining the minimum field of view of the target, since the field of view of the 3D laser scanner has three axes, namely the X-axis, Y-axis, and Z-axis, with the X-axis pointing to the right, the Y-axis pointing upwards, and the Z-axis pointing forwards, the maximum and minimum values ​​of the coordinates on the X-axis, Y-axis, and Z-axis are selected respectively. Finally, the extreme coordinates of the target weld are obtained, and the range covered by these extreme coordinates can encompass the target weld.

[0038] like Figure 3 As shown, after the target extreme coordinates are determined, a portion of the image is cropped from the overall color image according to the range (region of interest) involved by the target extreme coordinates. This cropped portion is then identified as the target image corresponding to the region of interest. It should be noted that the regions of interest for target welds in other welded areas can be determined in the same way.

[0039] S103: Extract the positioning area image of the target weld from the target image according to preset rules.

[0040] Specifically, after determining the target image corresponding to the region of interest of the target weld, the target image is sequentially processed by image filtering, grayscale conversion, binarization, and dilation. Then, contour extraction is performed to obtain multiple contours, and the largest contour is selected as the precise contour of the target weld. After determining the precise contour of the target weld, it can be accurately located in the target image. The location region image corresponding to the precise contour of the target weld is then extracted from the target image, thereby obtaining a color image containing only the target weld. It should be noted that in this application, the processing order can be image filtering, grayscale conversion, binarization, dilation, and contour extraction. In other embodiments, the target image can be grayscale converted first, followed by image filtering, binarization, dilation, and finally contour extraction.

[0041] S104: Extract the target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data corresponding to the target weld, and perform principal component analysis on the target three-dimensional point cloud data to obtain the weld size of the target weld.

[0042] Specifically, such as Figure 4 As shown, after the location area image of the target weld is determined, the target 3D point cloud data of the target weld is extracted from the overall 3D point cloud data corresponding to the location area image. One feasible extraction method is as follows: Since the location area image is cropped from a color image taken by a color camera, the location area image corresponds to the color camera coordinate system, and the 3D point cloud data corresponds to the laser projector coordinate system. First, according to the coordinate transformation matrix between the camera coordinate system and the laser projector coordinate system, the location area image of the target weld is transformed from the camera coordinate system to the laser projector coordinate system to obtain the 3D spatial coordinates corresponding to the location area image. Then, the target 3D point cloud data corresponding to the target weld is extracted from the overall 3D point cloud data based on these 3D spatial coordinates. Finally, principal component analysis is performed on the target 3D point cloud data to obtain the weld size of the target weld, which includes the weld length, width, and depth. Specifically, the analysis is performed using principal component analysis. Principal component analysis (PCA) is a statistical method that transforms a set of potentially correlated variables into a set of linearly uncorrelated variables through orthogonal transformation. This transformed set of variables is called the principal components. It is applied to point cloud preprocessing, plane detection, normal vector calculation, dimensionality reduction, classification, and decompression. This is existing technology and will not be described in detail here.

[0043] The coordinate transformation matrix is ​​a matrix that transforms a coordinate system from one space to another; it is also a mapping matrix showing the numerical transformation of coordinate points before and after the coordinate system transformation. It should be noted that the coordinate transformation matrix is ​​a preset value.

[0044] It should be noted that in weld size inspection applications, the commonly used method is to acquire weld dimensions solely through a color camera based on image processing algorithms. This method only stores the weld length and width, resulting in the loss of weld depth and thus low accuracy in weld size inspection. Furthermore, the accuracy of weld dimensions obtained through manual measurement is also insufficient. The weld location and size calculation method disclosed in this application integrates the overall 3D point cloud data acquired by a color camera and laser projector with the overall color image acquired by the color camera, significantly improving the accuracy of weld size inspection results.

[0045] See Figure 5 This application discloses a flowchart illustrating a method for weld positioning and dimension calculation, which can be implemented using a computer program or run on a weld positioning and dimension calculation device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application, specifically including: S201: Acquire overall 3D point cloud data of each welded area using a laser projector and a color camera, and acquire overall color images of each welded area using a color camera.

[0046] For details, please refer to step S101, which will not be repeated here.

[0047] In one feasible implementation, before S201, the method further includes: determining whether each welded area is within the measurement range of the 3D laser scanner. Specifically, step S201 includes: if the welded area is within the measurement range of the 3D laser scanner, acquiring 3D point cloud data of the welded area using a laser projector and a color camera, and acquiring a color image of the welded area using the color camera.

[0048] Specifically, before acquiring the overall 3D point cloud data and overall color image of each welded area, it is necessary to determine whether each welded area is within the measurement range of the 3D laser scanner. If a "confirmation" message confirming that all areas are within the measurement range is received from the personnel's terminal, it is determined that each welded area is within the measurement range of the 3D laser scanner. If no "confirmation" message is received, it is determined that each welded area is not within the measurement range of the 3D laser scanner. Therefore, personnel need to adjust the installation position of the 3D laser scanner according to its field of view and the size of each welded area to ensure that all welded areas are within its measurement range.

[0049] S202: Extract the target image corresponding to the region of interest from the overall color image. The region of interest is the region of interest of the target weld in the corresponding welded area.

[0050] For details, please refer to step S102, which will not be repeated here.

[0051] S203: Perform image segmentation, grayscale conversion, binarization, dilation processing, and contour extraction on the target image to obtain the maximum contour corresponding to the target weld.

[0052] S204: Based on the maximum contour, extract the positioning area image of the target weld from the target image.

[0053] In one feasible implementation, the target image is segmented using a preset image segmentation algorithm to obtain multiple segmented images; Each segmented image is sequentially subjected to grayscale conversion, binarization, dilation processing, and contour extraction to obtain multiple contours; Calculate the perimeter of each contour and determine the contour with the largest perimeter as the maximum contour corresponding to the target weld.

[0054] Specifically, such as Figure 6As shown, after the target image of the target weld is determined, image segmentation is first performed on the target image, mainly through an edge-based segmentation algorithm, which is essentially a filtering algorithm. In this embodiment, the Canny algorithm can be used, while in other embodiments, the Sobel algorithm can also be used. After image segmentation, multiple segmented images are obtained. Then, each segmented image is sequentially subjected to grayscale conversion, binarization, dilation, and contour extraction. One feasible grayscale conversion method is to use the cvtColor function based on OpenCV to perform grayscale conversion on each segmented image. Grayscale conversion simplifies the matrix and improves the calculation speed. Since the segmented image is a color image, the grayscale image after grayscale conversion occupies less memory and has a faster calculation speed. In addition, OpenCV is an open-source computer vision and machine learning software library. It contains many functions for processing images and videos and performing computer vision tasks such as image segmentation, object detection, edge detection, and shape analysis.

[0055] After grayscale conversion, binarization is performed. One feasible binarization method is to use the `im2bw` function in MATLAB. In other embodiments, the `threshold` function in OpenCV can also be used. Binarization is the process of converting an image into a binary image.

[0056] After binarization, dilation is performed. One feasible dilation method is to use a preset dilation algorithm. Dilation is a basic morphological processing algorithm commonly used in image processing and analysis. Dilation increases the target feature values, resulting in overall magnification of the target image. In this embodiment, the purpose of dilation is to remove noise such as weld slag and small particles around the target weld, thereby achieving more accurate positioning of the target weld.

[0057] After dilation, the corresponding contours are extracted from each binarized and dilated binary image using the `findContours` function in OpenCV, resulting in the contours for each segmented image. In other embodiments, contour extraction can also be performed using the `bwperim` function in MATLAB. Finally, the perimeter of multiple contours is calculated using the preset `arcLength` function. Since contours with smaller perimeters may contain noise such as weld slag, affecting the accurate positioning of the target weld, contours with smaller perimeters are discarded, and the contour with the largest perimeter is selected as the maximum contour of the target weld, thus enabling accurate positioning of the target weld. After determining the maximum contour, the positioning region image of the target weld is extracted from the target image corresponding to the target weld based on the maximum contour.

[0058] S205: Obtain the coordinate transformation matrix between the first coordinate system corresponding to the color camera and the second coordinate system corresponding to the laser projector.

[0059] S206: Based on the coordinate transformation matrix, the positioning area image of the target weld is transformed from the first coordinate system to the second coordinate system to obtain the target position information of the positioning area image. The target position information represents the relative positional relationship of the positioning area image to the overall color image.

[0060] S207: Based on the target location information, extract the target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data corresponding to the target weld.

[0061] Specifically, after the positioning area image is determined, the internal parameters of the 3D laser scanner are read from the factory manual. These internal parameters include the coordinate transformation matrix between the first coordinate system (color camera coordinate system) corresponding to the color camera and the second coordinate system (laser projector coordinate system) corresponding to the laser projector. The color camera coordinate system is a three-dimensional Cartesian coordinate system with the focal center of the color camera as the origin and the optical axis as the Z-axis. The laser projector coordinate system refers to the coordinate system through which the light rays projected by the laser projector pass. The origin of this coordinate system is on the plane of the laser projector, the x and y axes point to the projector lens, and the z-axis points to the bottom of the laser projector plane.

[0062] Since the target weld's location area image is in the first coordinate system, after the coordinate transformation matrix is ​​determined, the target weld's location area image is transformed from the first coordinate system to the second coordinate system (located in three-dimensional space). This yields the target position information of the location area image, which involves converting the pixels of the location area image to three-dimensional space. This target position information can be understood as including the three-dimensional coordinates corresponding to the pixels of the location area image, representing the relative position of the location area image within the overall color image. Finally, based on the target position information, the target three-dimensional point cloud data corresponding to the target weld is filtered out from the overall three-dimensional point cloud data.

[0063] S208: Perform principal component analysis on the target 3D point cloud data to obtain the weld size of the target weld.

[0064] For details, please refer to step S104, which will not be repeated here.

[0065] The implementation principle of the weld positioning and size calculation method in this application is as follows: After acquiring the overall three-dimensional point cloud data and overall color image of multiple welded areas, the region of interest (ROI) of the target weld in each welded area is determined, and the target image corresponding to this ROI is extracted from the overall color image of the welded area. This facilitates further analysis of the ROI and helps in the positioning of the target weld. Next, various noise interferences are eliminated, and the positioning area image of the target weld is accurately extracted from the target image. Finally, based on the conversion relationship between the color camera and the laser projector, the target three-dimensional point cloud data corresponding to the pixels of the positioning area image is extracted from the overall three-dimensional point cloud data corresponding to the welded area. Principal component analysis of the target three-dimensional point cloud data is performed, and the weld size of the target weld is finally determined. This eliminates the need for manual inspection, thereby improving the accuracy of the weld size detection results.

[0066] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0067] Please see Figure 7 This is a schematic diagram of the weld positioning and dimension calculation device provided in an embodiment of this application. This weld positioning and dimension calculation device can be implemented as all or part of a device through software, hardware, or a combination of both. The device 1 includes an information acquisition module 11, an image extraction module 12, a weld positioning module 13, and a dimension determination module 14.

[0068] The information acquisition module (11) is used to acquire the overall three-dimensional point cloud data of each welded area through a laser projector and a color camera, and to acquire the overall color image of each welded area through a color camera. The image extraction module (12) is used to extract the target image corresponding to the region of interest from the overall color image. The region of interest is the region of interest of the target weld in the corresponding welded area. The weld positioning module (13) is used to extract the positioning area image of the target weld from the target image according to preset rules; The size determination module (14) is used to extract the target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data corresponding to the target weld, and perform point cloud principal component analysis on the target three-dimensional point cloud data to obtain the weld size of the target weld.

[0069] Optionally, the image extraction module 12 specifically includes: Obtain the target minimum field of view corresponding to the target weld. The target minimum field of view is the minimum field of view required for the target weld to be fully scanned by a 3D laser scanner. The maximum and minimum values ​​of the X-axis, Y-axis and Z-axis directions of the minimum field of view of the target are selected respectively to obtain the target extreme value coordinates corresponding to the target weld. The target extreme value coordinates cover the target weld. The target image corresponding to the region of interest is obtained by cropping the image within the extreme coordinates of the target from the overall color image.

[0070] Optional, the weld positioning module 13 specifically includes: The target image is segmented, converted to grayscale, binarized, dilated, and its contour is extracted to obtain the maximum contour corresponding to the target weld. Based on the maximum contour, extract the positioning area image of the target weld from the target image.

[0071] Optional, the weld positioning module 13 specifically includes: The target image is segmented using a preset image segmentation algorithm to obtain multiple segmented images; Each segmented image is sequentially subjected to grayscale conversion, binarization, dilation processing, and contour extraction to obtain multiple contours; Calculate the perimeter of each contour and determine the contour with the largest perimeter as the maximum contour corresponding to the target weld.

[0072] Optional, the size determination module 14 specifically includes: Obtain the coordinate transformation matrix between the first coordinate system corresponding to the color camera and the second coordinate system corresponding to the laser projector; Based on the coordinate transformation matrix, the positioning area image of the target weld is transformed from the first coordinate system to the second coordinate system to obtain the target position information of the positioning area image. The target position information is the relative positional relationship between the positioning area image and the overall color image. Based on the target location information, extract the target 3D point cloud data corresponding to the positioning area image from the overall 3D point cloud data corresponding to the target weld; Principal component analysis of the target 3D point cloud data is performed to obtain the weld size of the target weld.

[0073] Optional, such as Figure 8 As shown, the device 1 also includes a pose determination module 15, which specifically includes: Determine whether each welded area is within the measurement range of the 3D laser scanner; The overall 3D point cloud data of each welded area is acquired using a laser projector and a color camera, and the overall color image of each welded area is also acquired using a color camera. Specifically, this includes: With each welded area within the measurement range of the 3D laser scanner, the overall 3D point cloud data of each welded area is acquired using a laser projector and a color camera, and the overall color image of each welded area is acquired using a color camera.

[0074] It should be noted that the weld positioning and dimension calculation device provided in the above embodiments is only illustrated by the division of the above functional modules when performing the weld positioning and dimension calculation method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the equipment can be divided into different functional modules to complete all or part of the functions described above. In addition, the weld positioning and dimension calculation device and the weld positioning and dimension calculation method embodiment provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiment, which will not be repeated here.

[0075] This application also discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it employs a weld positioning and dimension calculation method as described in the above embodiments.

[0076] The computer program can be stored in a computer-readable medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain middleware. The computer-readable medium includes any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the computer-readable medium includes, but is not limited to, the above-mentioned components.

[0077] The above-described method for weld positioning and dimension calculation is stored in the computer-readable storage medium and loaded and executed on the processor to facilitate the storage and application of the method.

[0078] This application also discloses an electronic device in which a computer program is stored in a computer-readable storage medium. When the computer program is loaded and executed by a processor, it employs the aforementioned method for weld positioning and dimension calculation.

[0079] The electronic device can be a desktop computer, a laptop computer, or a cloud server, and the electronic device includes, but is not limited to, a processor and a memory. For example, the electronic device may also include input / output devices, network access devices, and buses.

[0080] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it.

[0081] The memory can be an internal storage unit of an electronic device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) equipped on the electronic device. Furthermore, the memory can be a combination of an internal storage unit and an external storage device. The memory is used to store computer programs and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0082] In this electronic device, the weld positioning and dimension calculation method of the above embodiment is stored in the memory of the electronic device and loaded and executed on the processor of the electronic device for convenient use.

[0083] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for weld positioning and dimension calculation, characterized in that, Applied to a 3D laser scanner, the 3D laser scanner being equipped with a color camera and a laser projector, the method includes: The laser projector and the color camera are used to acquire overall three-dimensional point cloud data of each welded area, and the color camera is used to acquire overall color images of each welded area. Extracting the target image corresponding to the region of interest from the overall color image includes: obtaining the target minimum field of view corresponding to the target weld, wherein the target minimum field of view is the minimum field of view required for the target weld to be fully scanned by the three-dimensional laser scanner; The maximum and minimum values ​​of the X-axis, Y-axis and Z-axis directions of the minimum field of view of the target are selected respectively to obtain the target extreme coordinates corresponding to the target weld, and the target extreme coordinates cover the target weld; The image within the target extreme coordinates is extracted from the overall color image to obtain the target image corresponding to the region of interest; the region of interest is the region of interest of the target weld in the corresponding welded area; The positioning area image of the target weld is extracted from the target image according to preset rules; Extracting target 3D point cloud data corresponding to the positioning area image from the overall 3D point cloud data corresponding to the target weld, and performing principal component analysis on the target 3D point cloud data to obtain the weld size of the target weld, includes: obtaining a coordinate transformation matrix between the first coordinate system corresponding to the color camera and the second coordinate system corresponding to the laser projector; transforming the positioning area image of the target weld from the first coordinate system to the second coordinate system according to the coordinate transformation matrix to obtain target position information of the positioning area image, wherein the target position information represents the relative positional relationship of the positioning area image to the overall color image; extracting target 3D point cloud data corresponding to the positioning area image from the overall 3D point cloud data corresponding to the target weld according to the target position information; and performing principal component analysis on the target 3D point cloud data to obtain the weld size of the target weld.

2. The weld positioning and dimension calculation method according to claim 1, characterized in that, The step of extracting the positioning area image of the target weld from the target image according to preset rules specifically includes: The target image is segmented, converted to grayscale, binarized, dilated, and its contour is extracted to obtain the maximum contour corresponding to the target weld. Based on the maximum contour, the positioning area image of the target weld is extracted from the target image.

3. The weld positioning and dimension calculation method according to claim 2, characterized in that, The step of performing image segmentation, grayscale conversion, binarization, dilation processing, and contour extraction on the target image to obtain the maximum contour corresponding to the target weld seam specifically includes: The target image is segmented using a preset image segmentation algorithm to obtain multiple segmented images; Each segmented image is sequentially subjected to grayscale conversion, binarization, dilation processing, and contour extraction to obtain multiple contours; Calculate the perimeter of each of the aforementioned contours, and determine the contour with the largest perimeter as the maximum contour corresponding to the target weld.

4. The weld positioning and dimension calculation method according to claim 1, characterized in that, The step of performing principal component analysis on the target 3D point cloud data to obtain the weld size of the target weld specifically includes: The target three-dimensional point cloud data is analyzed using a preset principal component analysis method to obtain the weld size of the target weld.

5. The weld positioning and dimension calculation method according to claim 1, characterized in that, Before acquiring the overall three-dimensional point cloud data of each welded area using the laser projector and the color camera, the method further includes: Determine whether each welded area is within the measurement range of the three-dimensional laser scanner; The laser projector and the color camera acquire overall three-dimensional point cloud data of each welded area, and the color camera acquires overall color images of each welded area, specifically including: With each welded area within the measurement range of the 3D laser scanner, the overall 3D point cloud data of each welded area is acquired by the laser projector and the color camera, and the overall color image of each welded area is acquired by the color camera.

6. A weld positioning and dimension calculation device, used to implement the weld positioning and dimension calculation method according to any one of claims 1 to 5, characterized in that, include: The information acquisition module (11) is used to acquire the overall three-dimensional point cloud data of each welded area through a laser projector and a color camera, and to acquire the overall color image of each welded area through the color camera; Image extraction module (12) is used to extract the target image corresponding to the region of interest from the overall color image, wherein the region of interest is the region of interest of the target weld in the corresponding welded area; The weld positioning module (13) is used to extract the positioning area image of the target weld from the target image according to preset rules; The size determination module (14) is used to extract the target three-dimensional point cloud data corresponding to the positioning area image from the overall three-dimensional point cloud data corresponding to the target weld, and perform point cloud principal component analysis on the target three-dimensional point cloud data to obtain the weld size of the target weld.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs the method described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor loads and executes the computer program, it employs the method described in any one of claims 1-5.

Images