An open car side wall overhauling precision positioning and cutting method based on laser scanning image

Abstract

The present application relates to laser cutting positioning technical field, specifically to a kind of based on laser scanning image's open wagon side wall overhaul accurate positioning cutting method.S1: the point cloud data relevant to the side wall scanning of open wagon body, according to point cloud data to establish point cloud image;S2: according to the depth information of adjacent pixel edge extraction is carried out to point cloud image, the edge position extracted is marked as suspected weld edge position, and the point cloud data amount of non-suspected weld edge position is reduced, to obtain modeling image;S3: the image features of modeling image are identified, according to image features and the pose information of each frame modeling image, modeling image is grouped, and each group dense point cloud model is established, and the body side wall dense point cloud model is obtained by fusing each group dense point cloud model;S4: after grouping, plane fitting is carried out to point cloud data, according to the coordinate information of the intersection line of each type of plane, the weld pixel coordinates of weld position are determined.The precision of the side wall weld position positioning of open wagon body can be improved.

Description

Technical Field

[0001] This invention relates to the field of laser cutting and positioning technology, and specifically to a method for precise positioning and cutting of open wagon sidewalls for inspection based on laser scanning images. Background Technology

[0002] In my country's transportation industry, railway transportation accounts for a large proportion due to its advantages such as large cargo capacity and low freight rates. Open wagons are a type of railway freight car, with an all-steel welded structure consisting of a chassis, side walls, end walls, and doors. This type of wagon is mainly used for transporting bulk goods such as coal or ballast that are not susceptible to weathering. However, the corrosion of the wagon body caused by transporting goods, as well as wear, oxidation, and corrosion from prolonged use, pose serious risks to its operation, necessitating the replacement of steel plates from the side walls.

[0003] For cutting the sidewall steel plates of a vehicle body, the weld seams are used as a reference. Since these weld seams are difficult to identify, precise identification and positioning of the weld seams, while maintaining sufficient accuracy, is crucial to minimize damage to the open vehicle body during cutting and reduce costs. Currently, with the development of automation technology, the demand for precise weld seam positioning and cutting is becoming increasingly apparent, but the following technical challenges remain.

[0004] 1. The complexity of the working conditions of the vehicle body side walls: Due to the structural problems of the vehicle body side walls and end walls themselves, the steel plates themselves have problems such as bulging deformation, steel plate tearing, steel plate holes, and diagonal brace deformation. The working conditions of each side wall are different, making it difficult to achieve a standardized cutting process. Therefore, it is necessary to perform laser scanning on the side walls and establish a special model for different side walls to achieve accurate positioning.

[0005] 2. Currently, most vehicle body sidewall repair and cutting is done manually. While manual cutting can handle complex working conditions, the results often significantly increase the difficulty of subsequent repair and welding. Manual cutting leads to various adverse effects, such as insufficient flatness of the steel plate, high surface groove depth, and high degree of edge uniformity. Moreover, the cutting effect largely depends on the experience of the repair workers, making it difficult to achieve a consistent result. This approach cannot meet the needs of automation, process streamlining, and standardization of results in vehicle body sidewall cutting. In contrast, point cloud images scanned by industrial cameras are directly related to the robot's pose. During the scanning process, errors caused by changes in the robot's pose will have a certain impact on the final accuracy of point cloud registration. Summary of the Invention

[0006] The present invention aims to provide a precise positioning and cutting method for the inspection of open wagon sidewalls based on laser scanning images, so as to improve the accuracy of locating the weld seams of the sidewalls and endwalls of open wagons and improve the cutting quality.

[0007] The basic solution provided by this invention is a precise positioning and cutting method for inspecting the side wall of an open wagon based on laser scanning images, comprising the following steps:

[0008] S1: Perform laser scanning on the side wall of the open wagon body to obtain relevant point cloud data. The point cloud data includes the coordinate information, depth information and pose data of each pixel point during scanning. Establish a point cloud image based on the point cloud data.

[0009] S2: Extract edges from the point cloud image based on depth information, mark the extracted edge positions as suspected weld edge positions, and reduce the amount of point cloud data for non-suspected weld edge positions to obtain the modeling image;

[0010] S3: Identify the image features of the modeling images, group the modeling images according to the image features and the pose information of each frame of the modeling images, establish dense point cloud models of each group, and fuse the dense point clouds of each group to obtain the dense point cloud model of the vehicle side wall.

[0011] S4: Based on the depth information of the point cloud data in the dense point cloud model, the point cloud data is grouped and then fitted with planes to obtain various types of planes. Based on the coordinate information of the intersection lines of various types of planes, the weld pixel coordinates of the weld position are determined.

[0012] The principle of this invention is as follows: First, point cloud data of the open vehicle body is acquired. By comprehensively considering the depth information difference between adjacent pixels and the image orientation features of the weld joint, the laser-scanned image is processed a second time. Edge extraction is performed on the acquired point cloud data to determine the edge position of the weld, thereby determining the approximate position of the weld. Then, the positions of suspected weld pixels are considered to reduce the amount of point cloud data at non-suspected weld positions. The point cloud data at suspected weld positions is retained to reduce the amount of computational data, improve the accuracy of point cloud reconstruction, and reduce the accumulated error in the positioning process. By combining point cloud data fusion and plane fitting technology, the precise positioning of the weld position is achieved.

[0013] Compared to existing technologies, this invention directly performs secondary processing on the laser-scanned point cloud image, utilizing image processing methods to locate the weld seam. This approach achieves faster speed, lower computational load, and better timeliness while maintaining accuracy. During the 3D modeling of the open wagon's sidewall, digital image processing technology is combined to reduce the amount of data fusion and improve the timeliness of data processing. Simultaneously, based on the spatial and visual characteristics of the scanned image, accumulated errors during the positioning process are reduced, improving the accuracy of weld seam positioning.

[0014] Furthermore, S1 includes the following steps:

[0015] S100: Place a QR code at the center of the side wall of the open vehicle body, and scan the QR code to determine the reference position of the global coordinate system.

[0016] S110: Perform laser scanning on the entire sidewall of the open wagon body to obtain relevant point cloud data;

[0017] S120: Preprocess the point cloud data of each frame of the image to reduce noise and obtain the point cloud image.

[0018] A QR code was placed at the center of the vehicle's side wall as a reference. Scanning the QR code determined the global coordinate system reference position of the 3D model. Then, the entire side wall was scanned in 3D to obtain relevant point cloud data. This data contains the coordinates and depth information of each pixel, as well as the pose data during scanning. Subsequent preprocessing, such as mean filtering, was performed on each point cloud image to reduce noise during image acquisition, resulting in a point cloud image.

[0019] Further, S130: Repeat steps S100-S120 under different light intensities.

[0020] During scanning, scanning is performed under different light intensities to minimize the impact of external factors such as light, rust, and reflection on the laser scanning imaging effect.

[0021] Furthermore, S2 includes the following steps:

[0022] S200: The point cloud image obtained in S1 is labeled as image A. Edge extraction is performed on image A. The edge position is extracted using the depth information of image A as a feature. The gradient of the depth information is calculated. The depth abrupt change part is extracted as the suspected weld edge position based on the depth information gradient.

[0023] S210: Extract the suspected weld seam edge location and nearby pixels to create a mask image;

[0024] S220: Perform an AND operation between image A and the inverse code of the mask pattern, then downsample to obtain image B;

[0025] S230: Perform a bitwise AND operation between image A and the mask pattern to obtain image C;

[0026] S240: Perform an OR operation on image B and image C to obtain image D, and use image D as the modeling image.

[0027] Edge extraction is performed on image A. Based on the depth information of adjacent points, areas with abrupt changes in depth information are identified as potential weld edge locations. A mask image is then created using these potential weld edge locations and nearby pixels. A bitwise AND operation is performed between image A and the inverse of the mask image. The resulting image represents the non-edge region and does not contain the potential weld edge locations. This non-edge region is downsampled to reduce the point cloud data, resulting in image B. Simultaneously, a bitwise AND operation is performed between image A and the mask image to obtain image C, which contains all pixels at the potential weld edge locations. A bitwise OR operation is performed between image C and image B to obtain image D. Compared to the original image A, image D retains the weld location point cloud data while reducing the amount of point cloud data in the non-edge region and the computational load for subsequent point cloud reconstruction, thus improving the speed of point cloud reconstruction.

[0028] Furthermore, S3 includes the following steps:

[0029] S300: Identify the image features of the modeling image and classify the modeling image into three categories based on the image features. The image features include images containing diagonal braces, images containing columns, and images without columns and diagonal braces.

[0030] S310: Based on the three categories, the modeling images are grouped according to the pose data, and a visual bag-of-words model is established based on the image features of each group;

[0031] S320: Based on each set of modeling images and corresponding pose data, establish a dense point cloud model for each set of fused and reconstructed images.

[0032] S330: Determine whether the depth information difference between each frame of modeling image and dense point cloud model is within the set threshold. If yes, proceed to step S350; otherwise, proceed to step 340.

[0033] S340: Based on the density geometric consistency and photometric consistency, the depth and pose information of each frame of the modeling image in the group are corrected, and then step S320 is executed again.

[0034] S350: Based on the features of each group of visual bag-of-words models and the reconstruction depth model, perform fusion reconstruction, and determine whether the difference in depth information between each group of reconstruction models and the overall reconstruction model is within the set threshold. If yes, proceed to step S360; otherwise, proceed to step S350.

[0035] S360: Based on density consistency and photometric consistency, the depth and pose information of each frame modeling image in the group are corrected, and then step S340 is executed again.

[0036] S370: Obtain the dense point cloud model of the vehicle body sidewall.

[0037] In 3D modeling, coordinate transformation is performed on the scanned point cloud data using a reference position and position data during the scanning process, superimposing multiple frames of images acquired at different times, angles, and illuminations into the same coordinate system. Since the scanned point cloud image is directly related to the pose during scanning, and pose changes during scanning and image stitching can gradually accumulate errors, it affects the final accuracy of point cloud registration. Therefore, in this invention, images are first classified according to their features: images containing braces, images containing columns, and images without columns or braces. Based on these three categories, several frames are grouped according to the pose data corresponding to each frame, and a bag-of-words model is established based on the image features of each group, ensuring similarity between image features and pose data within each group. Then, loop closure detection is used to reduce accumulated errors, and the similarity between the current position and historical positions is used to correct the camera pose. In this invention, the similarity between the current position of each frame image and the position of the reconstructed dense point cloud is used to correct the position and reduce the camera pose accumulation error. Based on the similarity between the features and poses of each frame image after grouping, the coordinates of a group of images are transformed and fused for reconstruction. In each group, based on the density geometric consistency and photometric consistency, the position information of each frame image in the group is compared with the reconstructed dense point cloud. The difference between the data points of each frame image and the position of the reconstructed dense point cloud is used as feedback to optimize the depth of each frame image. This process is iterated until the position difference between the reconstructed dense point cloud and each frame image is within a certain threshold. Then, the similarity of the bag-of-words model between groups is used to fuse the dense point clouds established in each group. The previous steps are repeated to finally establish the final dense point cloud of the vehicle side wall and the reference position pose data, thus obtaining the final dense point cloud model of the vehicle side wall.

[0038] Furthermore, S4 includes the following steps:

[0039] S400: Based on the dense point cloud model of the vehicle side wall, point cloud data with the same depth information are grouped into one group, and point cloud data with significant changes in depth information are grouped into another group.

[0040] S410: Perform plane fitting on the grouped point cloud data. The fitted plane includes the steel plate platform, the inclined brace elevation plane, the inclined brace wing plane, the inclined brace wing weld plane, the side wall column plane, the side wall column wing weld plane, and the side wall column wing plane.

[0041] S420: Determine the pixel coordinates of the weld on one side of the vertical surface of the inclined brace and the weld on one side of the inclined brace based on the intersection of the plane of the steel plate and the plane of the vertical surface of the inclined brace and the intersection of the plane of the wing surface of the inclined brace and the plane of the weld on the wing surface of the inclined brace.

[0042] S430: Determine the pixel coordinates of the weld on one side of the diagonal brace facade and the weld on one side of the diagonal brace facade based on the intersection of the steel plate plane and the column facade plane and the column flange plane weld plane.

[0043] In the reconstructed dense point cloud model of the vehicle side wall, it is necessary to determine the pixel coordinates of the weld location. Since S2 filters the point cloud data, the point cloud data for the weld location in the final dense point cloud model of the side wall is preserved intact. Based on the established dense point cloud model of the side wall, point cloud data with the same depth are grouped into one category, and point cloud data with significant changes in depth information are grouped into another category. Plane fitting is then performed on the grouped data points. The fitting results include six planes: steel plate plane, brace elevation plane, brace wing plane, brace wing weld plane, side wall column plane, side wall column wing weld plane, and side wall column wing plane. The final weld location is determined based on the intersection lines of these planes. The welds on the side walls are mainly located near the diagonal braces and columns. There is a certain height difference between the pixels at the edge of the welds. The three-dimensional position information of each pixel can be obtained from the point cloud image. Plane fitting is performed on the three-dimensional data near the welds to obtain the intersection line of the two planes, which is the weld position. The cutting height is determined based on the depth information of the image, and the final cutting path is planned.

[0044] Furthermore, in step S1, an industrial laser camera with a calibration accuracy of 0.1 mm is used for laser scanning, and the pose information is the pose information of the industrial laser camera during scanning.

[0045] Laser scanning is performed using an industrial laser camera with a calibration accuracy of 0.1 mm to improve scanning precision. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating an embodiment of a precise positioning and cutting method for inspecting the side wall of an open wagon based on laser scanning images, according to the present invention.

[0047] Figure 2 This is a physical diagram of the open wagon sidewall, representing an embodiment of the present invention's method for precise positioning and cutting of open wagon sidewalls based on laser scanning images.

[0048] Figure 3 This is a schematic diagram of the process of S2 in an embodiment of the present invention, which is a method for precise positioning and cutting of open wagon sidewalls based on laser scanning images.

[0049] Figure 4 This is a flowchart illustrating step S3 in an embodiment of the present invention, which describes a precise positioning and cutting method for inspecting the side wall of an open wagon based on laser scanning images.

[0050] Figure 5This is a schematic diagram of a dense point cloud model of the vehicle side wall established in an embodiment of a method for precise positioning and cutting of open vehicle side wall based on laser scanning images according to the present invention.

[0051] Figure 6 This is a schematic diagram of the cutting path of the side wall of a wagon, established in an embodiment of the present invention, which is a method for precise positioning and cutting of the side wall of an open wagon based on laser scanning images. Detailed Implementation

[0052] The following detailed description illustrates the specific implementation method:

[0053] The basic implementation examples are as follows: Figure 1 As shown:

[0054] A precise positioning and cutting method for inspecting the sidewall of an open wagon based on laser scanning images includes the following steps:

[0055] S1: Perform laser scanning on the side wall of the open wagon body to obtain relevant point cloud data. The point cloud data includes the coordinate information, depth information and pose data of each pixel point during scanning. Establish a point cloud image based on the point cloud data.

[0056] S2: Extract edges from the point cloud image based on the depth information of adjacent pixels, mark the extracted edge positions as suspected weld edge positions, and reduce the amount of point cloud data at non-suspected weld edge positions to obtain the modeling image;

[0057] S3: Identify the image features of the modeling images, group the modeling images according to the image features and the pose information of each frame of the modeling images, establish dense point cloud models of each group, and fuse the dense point cloud models of each group to obtain the dense point cloud model of the vehicle side wall.

[0058] S4: Based on the depth information of the point cloud data in the dense point cloud model, the point cloud data is grouped and then fitted with planes to obtain various types of planes. Based on the coordinate information of the intersection lines of various types of planes, the weld pixel coordinates of the weld position are determined.

[0059] Open wagons are a type of railway freight car. They have an all-steel welded structure, consisting of a chassis, side walls, end walls, doors, and other components. Their main purpose is to transport bulk goods such as coal or ballast that are not susceptible to weathering. The side panel structure of an open wagon is as follows... Figure 2 As shown, it may have problems such as bulging deformation, steel plate tearing, steel plate holes, and diagonal brace deformation.

[0060] In step S1, a 0.1mm industrial laser camera is used to perform laser scanning on the side wall of the open vehicle body. The industrial laser camera is mounted on the robot arm to form a hand-eye system, which specifically includes the following steps:

[0061] S100: Place a QR code at the center of the side wall of the open wagon body, and scan the QR code to determine the reference position of the global coordinate system. Specifically, place a QR code at the center of the open wagon body to provide the center point for scanning and determine the reference position of the global coordinates of the 3D model.

[0062] S110: Perform a laser scan on the entire sidewall of the open vehicle to obtain relevant point cloud data. The relevant point cloud data includes the coordinate information, depth information, and robot pose data during the scan. The coordinate information refers to the coordinate position of each pixel in the global coordinate system, the depth information refers to the depth of each pixel relative to the same plane, and the pose data refers to the robot's posture, angle, and other poses during the scan.

[0063] S120: Preprocess the point cloud data of each frame of the image to reduce noise and obtain a point cloud image. Specifically, in this embodiment, preprocessing is performed by mean filtering to reduce the influence of noise in image acquisition and obtain a point cloud image.

[0064] S130: Repeat steps S100-S120 under different light intensities. Because an industrial laser camera with a calibration accuracy of 0.1mm is used to acquire vehicle information, the vehicle body needs to undergo anti-rust and anti-reflective treatments such as painting. Then, the industrial laser camera is used to scan under different light intensities. In this embodiment, 30%, 50%, and 80% light intensities are selected for laser scanning to minimize the impact of external factors such as light, rust, and reflections on the laser scanning imaging effect.

[0065] S2 is as follows: Figure 3 As shown, it includes the following steps:

[0066] S200: The point cloud image obtained in S1 is labeled as image A. Edge extraction is performed on image A. Using the depth information of image A as a feature, edge positions are extracted, the gradient of the depth information is calculated, and depth abrupt changes are extracted as suspected weld locations based on the depth information gradient. When performing edge extraction on image A, the depth information of image A is used as a feature. An appropriate edge extraction operator is used to calculate the gradient of the depth information. The depth information of the image is treated as a two-dimensional surface, and depth abrupt changes in the image are extracted. That is, when the depth information difference between adjacent pixels exceeds a certain threshold, it is considered a depth abrupt change. The depth abrupt changes are used as suspected weld locations.

[0067] S210: Extract the suspected weld seam edge location and nearby pixels to create a mask image. Specifically, the image masking operation refers to recalculating the value of each pixel in the image using a mask kernel operator. The mask kernel characterizes the influence of neighboring pixels on the new pixel value, and simultaneously performs a weighted average of the original pixels based on the weight factors in the mask operator. In this embodiment, the mask image is created based on pixel neighborhood traversal. Based on the neighborhood traversal, the source data matrix is ​​manipulated, the center target point is calculated with the current pixel position, the mask kernel operator template is moved pixel by pixel, the original image data is traversed, and the value of each pixel corresponding to the new image is updated.

[0068] S220: After performing a bitwise AND operation between image A and the inverse code of the mask image, downsampling is performed to obtain image B. The AND operation logic is 0 for all 0s and 1 for all 1s. Therefore, the image obtained by performing a bitwise AND operation between image A and the inverse code of the mask image is a non-edge region. This region does not contain weld information. Downsampling is performed on the non-edge region to reduce the amount of point cloud data, thus obtaining image B and reducing the amount of point cloud data in the non-edge region.

[0069] S230: Perform a bitwise AND operation between image A and the mask pattern to obtain image C. Image C contains pixels representing all suspected weld locations.

[0070] S240: Perform an OR operation on images B and C to obtain image D, and use image D as the modeling image. The OR operation logic is 1 if there is a 1 and 0 if there are all 0s. Using the obtained image D as the modeling image, compared with image A, while retaining the point cloud information of the weld location, it reduces the amount of point cloud data and the amount of calculation required for subsequent point cloud reconstruction, and improves the speed of subsequent point cloud reconstruction.

[0071] S3 is as follows: Figure 4 As shown, it includes the following steps:

[0072] S300: Identify the image features of the modeling image and classify the modeling image into three categories based on the image features. The image features include images containing diagonal braces, images containing columns, and images without columns and diagonal braces.

[0073] Specifically, the image features of the modeling images are identified, and the modeling images are divided into three categories based on these features. These image features include images containing diagonal braces, images containing columns, and images without columns or diagonal braces. First, the point cloud images are grouped to reduce the amount of data. Then, the modeling images obtained in S2 are classified using a clustering algorithm. By extracting the image features from each modeling image, the modeling images are divided into images containing diagonal braces, images containing columns, and images without columns or diagonal braces.

[0074] S310: Based on the three categories, the modeling images are grouped according to the pose data, and a visual bag-of-words model is established based on the image features of each group.

[0075] Based on these three categories, several frames are grouped according to the camera pose data corresponding to each frame. A bag-of-words (BOD) model is then built based on the features of each group. The BOD model is a technique for describing and calculating the similarity between images. It uses visual words to describe images by decomposing them into a set of independent features. These features consist of keypoints and descriptors. Keypoints and points of interest (POIs) are the same thing—points in certain spatial locations or within an image. These locations define the prominent parts of the image and are affected by factors such as image rotation, scaling, and translation. Descriptors are the values ​​of these keypoints. The clustering algorithm used when creating the dictionary is based on these descriptors. The image is traversed, and the presence of words is checked. If a word is found, its count is incremented. At this point, each group of data contains similar image features and pose data.

[0076] S320: Based on each set of modeling images and corresponding pose data, establish a dense point cloud model for each set of fused reconstructions.

[0077] S330: Determine whether the depth information difference between each frame of the modeling image and the dense point cloud model is within the set threshold. If yes, proceed to step S350; otherwise, proceed to step 340.

[0078] During modeling, loop closure detection is employed to reduce the cumulative error caused by robot pose changes and image stitching. Loop closure detection corrects camera pose data by detecting the similarity between the current position and historical positions. This invention summarizes this by using the similarity between the nitrogen position in each frame and the position of the reconstructed dense point cloud to correct the pose, reducing the accumulated camera pose error caused by the robot's hand-eye system movement during scanning. Based on the similarity between the image features and pose of each frame after grouping, the coordinates of the group's images are transformed and fused for reconstruction. Within each group, based on dense geometric consistency and photometric consistency, the positional information of each frame in the group is compared with the reconstructed dense point cloud. The difference between the positions of each data point in each frame and the reconstructed dense point cloud is used as feedback to optimize the depth map of each frame, continuously iterating until the positional difference between the reconstructed dense point cloud and each frame is within a certain threshold.

[0079] S340: Based on the density geometric consistency and photometric consistency, the depth and pose information of each frame of the modeling image in the group are corrected, and then step S320 is executed again.

[0080] S350: Based on the features of each group of visual bag-of-words models and the reconstruction depth model, perform fusion reconstruction, and determine whether the difference in depth information between each group of reconstruction models and the overall reconstruction model is within the set threshold. If yes, proceed to step S360; otherwise, proceed to step S350.

[0081] S360: Based on density consistency and photometric consistency, the depth and pose information of each frame modeling image in the group are corrected, and then step S340 is executed again.

[0082] S360: Obtain the dense point cloud model of the vehicle body sidewall.

[0083] By leveraging the similarity of the bag-of-words models among the groups, the dense point clouds built in each group are fused. The same reconstruction method is used for each group, ultimately creating the final dense point cloud of the vehicle sidewall and the baseline position and pose data for the corrective robot. The final reconstructed vehicle sidewall model is shown below. Figure 5 As shown.

[0084] S4 specifically includes the following steps:

[0085] S400: Based on the dense point cloud model of the vehicle side wall, point cloud data with the same depth information are grouped into one group, and point cloud data with significant changes in depth information are grouped into another group.

[0086] In the reconstructed dense point cloud model of the vehicle side wall, the pixel coordinates of the weld location need to be determined first. Since S2 filters the point cloud data, the point cloud data of the weld location in the final dense point cloud model of the side wall is preserved intact. Based on the established dense point cloud model of the side wall, point cloud data with the same depth are grouped into one category, and point cloud data with significant changes in depth information, i.e., changes exceeding a certain threshold, are grouped into another category. The point cloud registration and data fusion in S3 minimize the error in this step.

[0087] S410: Perform plane fitting on the grouped point cloud data. The fitted plane includes the steel plate platform, the inclined brace elevation plane, the inclined brace wing plane, the inclined brace wing weld plane, the side wall column plane, the side wall column wing weld plane, and the side wall column wing plane.

[0088] S420: Based on the intersection of the steel plate plane and the vertical plane of the brace and the intersection of the brace flange plane and the weld plane of the brace flange, determine the coordinate information of the weld on one side of the vertical plane of the brace and the weld on one side of the brace flange.

[0089] S430: Determine the coordinate information of the weld on one side of the diagonal brace facade and the weld on one side of the diagonal brace facade based on the intersection of the steel plate plane and the column facade plane and the column flange plane weld plane.

[0090] The welds on the side walls are mainly located near the diagonal braces and columns. There is a certain height difference between the pixels at the weld edges. The 3D position information of each pixel can be obtained from the point cloud image. Plane fitting is performed on the 3D data near the weld to obtain the intersection line of the two planes, which is the weld location. The cutting height is determined based on the image's depth information, and the final cutting path is planned. The final weld coordinates are as follows: Figure 6 As shown.

[0091] Finally, based on the determined pixel coordinates of the weld edge and the dense point cloud model of the side wall established in step three, the depth information of each pixel at the weld position is determined. Combined with the transfer matrix between the dense point cloud obtained in step three and the robot's reference position pose, the robot's pose data at each obtained weld coordinate is determined. This allows the robot to handle situations such as body tearing, bulging, and deformation, while ensuring that the cut steel plate is not deformed too much and that the trajectory after cutting is smooth, facilitating subsequent welded steel plate repair.

[0092] The above are merely embodiments of the present invention. It should be noted that the technical solutions in the above embodiments can be applied not only to the precise positioning and cutting of the sidewalls of open wagons, but also to the positioning and cutting of the endwalls of open wagons. Commonly known structures and characteristics in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are able to obtain all existing technologies in the field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can improve and implement this solution based on the guidance provided in this application and their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be pointed out that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for precise positioning and cutting of open wagon sidewalls for inspection based on laser scanning images, characterized in that: Includes the following steps: S1: Perform laser scanning on the side wall of the open wagon body to obtain relevant point cloud data. The point cloud data includes the coordinate information, depth information and pose data of each pixel point during scanning. Establish a point cloud image based on the point cloud data. S2: Extract edges from the point cloud image based on the depth information of adjacent pixels, mark the extracted edge positions as suspected weld edge positions, and reduce the amount of point cloud data at non-suspected weld edge positions to obtain the modeling image; S3: Identify the image features of the modeling images, group the modeling images according to the image features and the pose information of each frame of the modeling images, establish dense point cloud models of each group, and fuse the dense point cloud models of each group to obtain the dense point cloud model of the vehicle side wall. S4: Based on the depth information of the point cloud data in the dense point cloud model, the point cloud data is grouped and then fitted with planes to obtain various types of planes. Based on the coordinate information of the intersection lines of various types of planes, the weld pixel coordinates of the weld position are determined. S1 includes the following steps: S100: Place a QR code at the center of the side wall of the open vehicle body, and scan the QR code to determine the reference position of the global coordinate system. S110: Perform laser scanning on the entire sidewall of the open wagon body to obtain relevant point cloud data; S120: Preprocess the point cloud data of each frame of image to reduce noise and obtain point cloud image; S2 includes the following steps: S200: The point cloud image obtained in S1 is labeled as image A. Edge extraction is performed on image A. The edge position is extracted using the depth information of image A as a feature. The gradient of the depth information is calculated. The depth abrupt change part is extracted as the suspected weld position based on the depth information gradient. S210: Extract the suspected weld seam edge location and nearby pixels to create a mask image; S220: Perform an AND operation between image A and the inverse code of the mask pattern, then downsample to obtain image B; S230: Perform a bitwise AND operation between image A and the mask pattern to obtain image C; S240: Perform an OR operation on image B and image C to obtain image D, and use image D as the modeling image; S3 includes the following steps: S300: Identify the image features of the modeling image and classify the modeling image into three categories based on the image features. The image features include images containing diagonal braces, images containing columns, and images without columns and diagonal braces. S310: Based on the three types of modeling images, group the modeling images of each type according to the pose data, and establish a visual bag-of-words model based on the image features of each group; S320: Based on the modeling image of each frame in each group and the corresponding pose data, establish a dense point cloud model for each group of fused reconstructions. S330: Determine whether the depth information difference between each frame of modeling image and the dense point cloud model is within the set threshold. If yes, proceed to step S350; otherwise, proceed to step 340. S340: Based on the density geometric consistency and photometric consistency, the depth and pose information of each frame of the modeling image in the group are corrected, and then step S320 is executed again. S350: Based on the features of each group of visual bag-of-words models and the reconstruction depth model, perform fusion reconstruction, and determine whether the difference in depth information between each group of reconstruction models and the overall reconstruction model is within the set threshold. If yes, proceed to step S360; otherwise, proceed to step S350. S360: Based on density consistency and photometric consistency, the depth and pose information of each frame modeling image in the group are corrected, and then step S340 is executed again. S370: Obtain the dense point cloud model of the vehicle body sidewall.

2. The method for precise positioning and cutting of open wagon sidewalls based on laser scanning images according to claim 1, characterized in that: S1 further includes the following steps: S130: Repeat S100-S120 under different light intensities.

3. The method for precise positioning and cutting of open wagon sidewalls based on laser scanning images according to claim 1, characterized in that: S4 includes the following steps: S400: Based on the dense point cloud model of the vehicle side wall, point cloud data with the same depth information are grouped into one group, and point cloud data with significant changes in depth information are grouped into another group. S410: Perform plane fitting on the grouped point cloud data. The fitted plane includes the steel plate platform, the inclined brace elevation plane, the inclined brace wing plane, the inclined brace wing weld plane, the side wall column plane, the side wall column wing weld plane, and the side wall column wing plane. S420: Based on the intersection of the steel plate plane and the vertical plane of the brace and the intersection of the brace flange plane and the weld plane of the brace flange, determine the coordinate information of the weld on one side of the vertical plane of the brace and the weld on one side of the brace flange. S430: Determine the coordinate information of the weld on one side of the diagonal brace facade and the weld on one side of the diagonal brace facade based on the intersection of the steel plate plane and the column facade plane and the column flange plane weld plane.

4. The method for precise positioning and cutting of open wagon sidewalls based on laser scanning images according to claim 1, characterized in that: In step S1, an industrial laser camera with a calibration accuracy of 0.1 mm is used for laser scanning, and the pose information is the pose information of the industrial laser camera during scanning.

Images