A method and system for detecting a cutting point position of an arc extinguishing plate at a pipe end of a steel pipe

Abstract

A steel pipe end arc plate cutting point detection method and system relate to the field of steel pipe cutting positioning detection. The method solves the problems of low detection efficiency and poor detection stability in the existing steel pipe end arc plate cutting point detection technology. The method comprises the following steps: collecting arc plate point cloud data; clipping the point cloud data and processing the clipped point cloud data by a statistical method to obtain purified point cloud data; projecting the purified point cloud data according to an external parameter matrix to obtain a projected two-dimensional image and a depth image; preprocessing the projected two-dimensional image to obtain an arc plate contour; establishing a deep learning model and detecting the arc plate cutting point rectangular region through the deep model; and determining the steel pipe end arc plate cutting point according to the fitting interpolation and the arc plate cutting point rectangular region. The application is applied to the field of steel pipe cutting.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of steel pipe cutting positioning detection, especially to a steel pipe end arc plate cutting point detection method. BACKGROUND

[0002] The steel pipe end arc plate cutting point detection technology can ensure the correct installation and position of the arc plate, improve the safety of the steel pipe system, and prevent fire and explosion risks. This detection can help find possible defects, errors or unqualified cutting points, and the presence of damage or foreign matter, so that timely measures are taken to correct or repair, ensuring the normal operation and safety of the steel pipe system.

[0003] The existing steel pipe end arc plate cutting point detection technology includes: manual detection method, laser point positioning detection method, laser tracking detection method and 3D camera and image processing combined detection method.

[0004] However, in the existing steel pipe end arc plate cutting point detection technology, the manual detection method has the problems of high labor cost, low automation level, low detection accuracy and low detection efficiency; the laser point positioning and laser tracking detection methods have the problems of low detection efficiency, poor environmental adaptability and small detection range; the 3D camera and image processing combined detection method has the problems of low detection efficiency and poor detection stability.

[0005] If the detection accuracy is not high or the detection efficiency is not high, not only the maintenance cost will increase, but also safety hazards will be caused. If the arc plate cutting point cannot be accurately positioned and diagnosed, multiple repairs or maintenance may be required, increasing the maintenance cost and the complexity of maintenance. Low precision or low efficiency detection may fail to timely discover and solve the adverse conditions of the arc plate cutting point, which will also affect the quality of the steel pipe system and may cause system performance degradation or failure; inaccurate detection results may fail to discover or misjudge the defects or problems of the arc plate cutting point, thereby increasing the potential safety risk. SUMMARY

[0006] The present application proposes a steel pipe end arc plate cutting point detection method to solve the problems of low detection efficiency and poor detection stability in the existing steel pipe end arc plate cutting point detection technology. The specific scheme is as follows:

[0007] The steel pipe end arc plate cutting point detection method comprises the following steps:

[0008] S1: collect point cloud data of the arc extinguishing plate;

[0009] S2: clip the point cloud data, and process the clipped point cloud data by a statistical method to obtain purified point cloud data;

[0010] S3: project the purified point cloud data according to an external parameter matrix to obtain a projected two-dimensional image and a depth image;

[0011] S4: pre-process the projected two-dimensional image to obtain an arc extinguishing plate contour;

[0012] S5: establish a deep learning model, and detect a cutting point position rectangular region of the arc extinguishing plate by the deep model;

[0013] S6: determine a pipe end arc extinguishing plate cutting point position according to fitting interpolation and the cutting point position rectangular region of the arc extinguishing plate.

[0014] Further, a preferred mode is also provided, wherein the step S1 comprises:

[0015] A 3D camera is installed on a six-axis robot flange, and a camera external parameter matrix is determined by a robot hand-eye calibration method;

[0016] A 3D camera is triggered by communication with the robot through TCP, and point cloud arc extinguishing plate data is obtained by photographing in the camera external parameter matrix.

[0017] Further, a preferred mode is also provided, wherein the step S4 comprises:

[0018] A two-dimensional image after projection is subjected to convolution filter processing by a median filter algorithm and a Gaussian filter algorithm;

[0019] Edge contour information of the two-dimensional image after filter processing is extracted by using a Canny algorithm;

[0020] A binary image of the edge contour information is obtained by Otsu method;

[0021] An arc extinguishing plate contour is obtained by extracting an image main body contour in the binary image according to eight-neighborhood of a pixel point and removing isolated noise points and contour noise.

[0022] Further, a preferred mode is also provided, wherein the deep model of the S5 is established by:

[0023] Mosaic data enhancement is removed from an input end of a YOLOv5s network, and a new data enhancement method is added to the input end;

[0024] A backbone network of the YOLOv5s network is subjected to null processing;

[0025] The new data enhancement method is: image random rotation, flipping, translation and modification;

[0026] Removing the 20x20 feature map layer from the output end of the YOLOv5s network;

[0027] According to the removed output end, the associated CBL, CSP, Concat and Conv modules in the neck network of the YOLOv5s network are removed;

[0028] According to the modified input end, the main network, the modified neck network and the modified output end, a deep model is constructed.

[0029] Further, a preferred way is also provided, wherein the modification is to modify the size of the input image to 320x320x1.

[0030] Further, a preferred way is also provided, wherein the step S6 comprises:

[0031] Step 1: Obtain the maximum value point of each row of pixels in the cutting point position rectangular region of the arc extinguishing plate and the number of maximum value points, and obtain the horizontal straight line expression according to the maximum value points;

[0032] Step 2: Extract the maximum value point of each column of pixels in the cutting point position rectangular region of the arc extinguishing plate and the number of maximum value points, and obtain the vertical straight line expression;

[0033] Step 3: Determine the intersection point coordinates and the cutting point position coarse positioning coordinates according to the horizontal straight line expression and the vertical straight line expression;

[0034] Step 4: According to the cutting point position coarse positioning coordinates, a full zero mask is made and overlaid to the cutting point position rectangular region image, and steps 1 to 3 are repeated to obtain the mask positioning coordinates;

[0035] Step 5: Take the mask positioning coordinates as the center, traverse the pixel points, and end the traversal when the preset condition is met to determine the mask cutting point position fine positioning coordinates; if there is no coordinate point meeting the condition, the mask cutting point position fine positioning coordinates are the initial obtained coordinates;

[0036] Step 6: Obtain the cutting point position fine positioning coordinates and depth information of the arc extinguishing plate according to the mask cutting point position fine positioning coordinates and the cutting point position rectangular region coordinate information;

[0037] Step 7: According to the cutting point position fine positioning coordinates and depth information of the arc extinguishing plate, the point cloud point position information is obtained by mapping.

[0038] Further, a preferred way is also provided, wherein the horizontal straight line expression comprises:

[0039] Extract the maximum value point and the number of maximum value points of each row of pixels in the cutting point position rectangular region of the arc extinguishing plate, retain if the number of maximum value points is less than 3, and determine the coordinate data according to the maximum value points;

[0040] Filter the coordinate points of the coordinate data to obtain filtered coordinate data;

[0041] Perform least square fitting processing on the filtered coordinate data to obtain a horizontal straight line expression.

[0042] Based on the same inventive concept, the application also provides a steel pipe end arc extinguishing plate cutting point position detection system, the system comprises:

[0043] A data acquisition module is configured to acquire point cloud data of the arc extinguishing plate.

[0044] A point cloud data cropping module is configured to crop the point cloud data and process the cropped point cloud data by using a statistical method to obtain purified point cloud data.

[0045] A projection module is configured to project the purified point cloud data according to an external parameter matrix to obtain a projected two-dimensional image and a depth image.

[0046] A preprocessing module is configured to preprocess the projected two-dimensional image to obtain an arc extinguishing plate contour.

[0047] A cutting point position rectangular region acquisition module is configured to establish a deep learning model and detect the cutting point position rectangular region of the arc extinguishing plate by using the deep model.

[0048] A pipe end arc extinguishing plate cutting point position acquisition module is configured to determine the pipe end arc extinguishing plate cutting point position according to fitting interpolation and the cutting point position rectangular region of the arc extinguishing plate.

[0049] Based on the same inventive concept, the application also provides a computer readable storage medium for storing a computer program, wherein the computer program executes any one of the steel pipe end arc extinguishing plate cutting point position detection methods.

[0050] Based on the same inventive concept, the application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes any one of the steel pipe end arc extinguishing plate cutting point position detection methods.

[0051] The application has the following advantages:

[0052] The application solves the problems of low detection efficiency and poor detection stability in the existing steel pipe end arc extinguishing plate cutting point position detection technology.

[0053] The present application can obtain the three-dimensional geometric information of the arc extinguishing plate, including shape and surface characteristics, by collecting point cloud data. By clipping and purifying the point cloud data, irrelevant areas and noise points can be removed, improving the accuracy and efficiency of subsequent processing. The point cloud data is projected into two-dimensional images and depth images, facilitating image processing and deep learning-based algorithm analysis and detection. By analyzing the two-dimensional images through preprocessing, the contour information of the arc extinguishing plate is extracted, providing accurate target areas for subsequent positioning and detection. Using a deep learning model for detection, the rectangular region of the arc extinguishing plate cutting point position can be automatically identified and positioned, improving the accuracy and efficiency of detection. By fitting interpolation and the information of the arc extinguishing plate cutting point position rectangular region, the cutting point position of the pipe end arc extinguishing plate is accurately positioned, improving the safety and operating efficiency of the pipeline system.

[0054] The present application is applied to the field of steel pipe cutting. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 A flow chart of a steel pipe end arc extinguishing plate cutting point position detection method according to embodiment one;

[0056] Figure 2 A schematic diagram of a preprocessed two-dimensional image according to embodiment three;

[0057] Figure 3 A schematic diagram of a deep learning model according to embodiment four;

[0058] Figure 4 A left cutting point position image schematic diagram according to embodiment eleven;

[0059] Figure 5 A right cutting point position image schematic diagram according to embodiment eleven. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments of the present application.

[0061] Embodiment one, see Figure 1 This embodiment describes a steel pipe end arc extinguishing plate cutting point position detection method. The method includes:

[0062] S1: Collecting arc extinguishing plate point cloud data;

[0063] S2: Clipping the point cloud data and using statistical methods to process the clipped point cloud data to obtain purified point cloud data;

[0064] S3: Projecting the purified point cloud data according to the extrinsic matrix to obtain a projected two-dimensional image and a depth image;

[0065] S4: Preprocessing the projected two-dimensional image to obtain the profile of the arc extinguishing plate;

[0066] S5: Establishing a deep learning model and detecting the rectangular region of the arc extinguishing plate cutting point position through the deep model;

[0067] S6: Determining the pipe end arc extinguishing plate cutting point position according to the fitting interpolation and the arc extinguishing plate cutting point position rectangular region.

[0068] The present embodiment can obtain the three-dimensional geometric information of the arc extinguishing plate, including shape and surface features, by collecting point cloud data. By clipping and purifying the point cloud data, irrelevant areas and noise points can be removed, improving the accuracy and efficiency of subsequent processing. Projecting the point cloud data into two-dimensional images and depth images facilitates image processing and deep learning-based algorithm analysis and detection. By preprocessing the two-dimensional image, the profile information of the arc extinguishing plate is extracted, providing accurate target regions for subsequent positioning and detection. Using a deep learning model for detection can automatically identify and locate the rectangular region of the arc extinguishing plate cutting point position, improving the accuracy and efficiency of detection. By fitting interpolation and the information of the arc extinguishing plate cutting point position rectangular region, the cutting point position of the pipe end arc extinguishing plate is accurately positioned, improving the safety and operating efficiency of the pipeline system.

[0069] Embodiment two, the present embodiment is a further limitation of the steel pipe end arc extinguishing plate cutting point position detection method of embodiment one, the step S1 comprises:

[0070] A 3D camera is installed on the flange of a six-axis robot, and the camera extrinsic matrix is determined by a robot hand-eye calibration method;

[0071] The 3D camera is triggered by communicating with the robot through TCP, and the point cloud arc extinguishing plate data is obtained by taking pictures within the camera extrinsic matrix.

[0072] The embodiment fixes the 3D camera on the flange of the six-axis robot, ensures the stability of the camera position and the non-collision with the motion trajectory of the robot, uses the robot hand-eye calibration method to determine the camera extrinsic matrix, that is, to determine the position and attitude relationship of the camera in the robot coordinate system, establishes a TCP / IP communication connection, and the computer or control system and the robot are communicated through the communication interface for data transmission and control command interaction. By sending a trigger command to the robot, the 3D camera is triggered to take a picture, and the camera takes an image according to the preset parameters at the specified position on the flange, and obtains point cloud data. Through the camera extrinsic matrix, the points in the robot coordinate system are converted into the camera coordinate system, and the picture is taken in the camera coordinate system. In this way, the point cloud data can be aligned with the robot coordinate system; according to the image data taken by the camera, the image is processed using the depth sensor or algorithm of the camera, and the point cloud data is extracted, which represents the three-dimensional geometric information of the arc extinguishing plate.

[0073] In the embodiment, the automatic photographing is realized by robot operation and camera trigger command, the errors and inconsistencies of manual operation are reduced, the extrinsic matrix of the camera is determined by the hand-eye calibration method, the position and attitude relationship of the camera in the robot coordinate system can be accurately determined, and the accuracy of the data is ensured. The high-speed motion ability and precise positioning ability of the robot can quickly obtain point cloud data, and the consistency can be maintained in repeated operations. The combination of the robot and the camera can adapt to the shooting requirements of the arc extinguishing plate at different positions and angles, and has high flexibility.

[0074] In the embodiment, the automatic photographing process is realized by the cooperation of the robot and the 3D camera, and the point cloud arc extinguishing plate data is obtained by the determined camera extrinsic matrix. The embodiment improves the speed and accuracy of data acquisition, and provides a reliable data basis for subsequent point cloud data processing and analysis.

[0075] Embodiment three, see Figure 2 The embodiment is a further limitation of the steel pipe end arc extinguishing plate cutting point detection method described in embodiment one. The step S4 includes:

[0076] The projected two-dimensional image is convoluted and filtered by a median filtering algorithm and a Gaussian filtering algorithm;

[0077] The edge profile information of the filtered two-dimensional image is extracted using a Canny algorithm;

[0078] A binary image of the edge profile information is obtained by Otsu method;

[0079] The main body profile of the binary image is extracted according to the eight-neighborhood of the pixel points, and isolated noise points and profile noise are removed, to obtain the arc extinguishing plate profile.

[0080] The embodiment first applies a median filtering algorithm and a Gaussian filtering algorithm to the projected two-dimensional image for convolution filtering processing. The median filtering algorithm eliminates noise by taking the median value of pixels in the neighborhood and retains edge information. The Gaussian filtering algorithm smoothes the image by weighted averaging of pixels in the neighborhood around the pixel, while also retaining edge information. Then, the Canny algorithm is applied to the filtered image to extract edge contour information through the following steps: grayscale processing of the image; applying Gaussian filtering to eliminate noise in the image; calculating the gradient amplitude and direction of the image; applying non-maximum suppression to refine the gradient amplitude along the edge direction; applying double threshold processing to classify edge pixels into strong edges and weak edges; using edge connection methods to retain strong edges above the high threshold and connect weak edges below the high threshold but connected to strong edges to form complete edge contours. Threshold segmentation is performed on the edge contour information obtained by the Canny algorithm to obtain a binary image. Otsu method automatically determines the optimal threshold according to the histogram of the image, maximizing the inter-class variance between the background and the foreground. Based on the connectivity of the eight-neighborhood of the pixel points, the main contour of the image is extracted by performing connected region analysis on the binary image. At the same time, according to the preset threshold, isolated noise points and contour noise are removed to ensure that the extracted edge contour accurately represents the shape of the arc plate, and the arc plate contour image l' is obtained, as shown in Figure 2

[0081] The median filtering and Gaussian filtering in the embodiment can effectively eliminate noise in the image and provide more accurate edge contour information; the Canny algorithm can detect edges in the image, including strong edges and weak edges, and connect them to form continuous edge contours; the Otsu method can automatically determine the optimal threshold based on statistical information of the image without manual setting. Through analysis of the eight-neighborhood of the pixel points, isolated noise points and contour noise can be removed, and more accurate and clear arc plate contours can be extracted.

[0082] The embodiment is to obtain accurate contour information of the arc plate after filtering and edge extraction processing, providing a reliable basis for subsequent arc plate detection and analysis.

[0083] Embodiment Four, see Figure 3 This embodiment is a further limitation of the steel pipe end arc plate cutting point detection method described in Embodiment One. The method for establishing the depth model of S5 is:

[0084] The input end of the YOLOv5s network is removed from the Mosaic data enhancement, and a new data enhancement method is added to the input end;

[0085] The backbone network of the YOLOv5s network is null processing;

[0086] ​The new data enhancement method is: image random rotation, flipping, translation and modification;

[0087] Removing the 20x20 feature map layer from the output end of the YOLOv5s network;

[0088] According to the removed output end, the associated CBL, CSP, Concat and Conv modules in the neck network of the YOLOv5s network are removed;

[0089] According to the modified input end, backbone network, modified neck network and modified output end, a deep model is constructed.

[0090] The present embodiment improves the existing deep learning infrastructure YOLOv5s, which is divided into four parts: input end, backbone network (Backbone), neck network (Neck) and output end (Prediction), wherein the backbone network and the neck network structure are mainly composed of CBL, FOCUS, CSP, SPP and Conv modules. Since the image l is an 8-bit single-channel binary image, and the image content is simple, the detection category is only four categories, in order to reduce the calculation complexity, improve the detection efficiency and detection accuracy, the original structure is modified, and the specific modification is as follows:

[0091] (1) Input end modification, remove Mosaic data enhancement method, use image random rotation, flipping and translation to enhance the data, and modify the input image shape to (320, 320, 1);

[0092] (2) Neck network and output end modification, since the size of the workpiece in image l' is relatively stable, the size of the detection target in the image is small, and at the same time, since the size of the detection region is large, the role of small-scale feature layer will be reduced, therefore, based on the original YOLOv5s, the 80x80 scale feature map layer in the output end structure and the associated CBL, CSP, Concat modules in the neck network (Neck) are removed, the 40x40 scale and 20x20 scale feature map layers in the output end and the associated modules in the neck network (Neck) are retained, to reduce the complexity of the model structure and further improve the model recognition speed. Since the input image shape has changed, without changing the backbone network (Backbone), the 80x80 scale feature map layer in the original output end becomes a 40x40 scale feature map layer, and the 40x40 scale feature map layer becomes a 20x20 scale feature map layer, therefore, only the 20x20 scale feature map layer in the original output end and the associated CBL, CSP, Concat and Conv modules in the neck network (Neck) need to be removed. The modified network structure is shown in Figure 3 .

[0093] Embodiment five, the embodiment is a further limitation of the steel pipe end arc plate cutting point detection method of embodiment four, the modification is: the size of the input image is modified to 320x320x1.

[0094] Specifically, the length of the input image is modified to 320 pixels, the width is modified to 320 pixels, and 1 channel is selected.

[0095] Embodiment six, the embodiment is a further limitation of the steel pipe end arc plate cutting point detection method of embodiment one, step S6 includes:

[0096] Step 1: Obtain the maximum value point of each row of pixels in the cutting point rectangular region of the arc plate and the number of maximum value points, and obtain the horizontal straight line expression according to the maximum value point;

[0097] Step 2: Extract the maximum value point of each column of pixels in the cutting point rectangular region of the arc plate and the number of maximum value points, and obtain the vertical straight line expression;

[0098] Step 3: Determine the intersection coordinates and the cutting point coarse positioning coordinates according to the horizontal straight line expression and the vertical straight line expression;

[0099] Step 4: According to the cutting point coarse positioning coordinates, make a full zero mask, and cover it to the cutting point rectangular region image, repeat steps 1 to 3, and obtain the mask positioning coordinates;

[0100] Step 5: Take the mask positioning coordinates as the center, traverse the pixel points, and end the traversal when the preset condition is met to determine the mask cutting point fine positioning coordinates; If there is no coordinate point meeting the condition, the mask cutting point fine positioning coordinates is the initial obtained coordinate;

[0101] Step 6: Obtain the cutting point fine positioning coordinates and depth information of the arc plate according to the mask cutting point fine positioning coordinates and the cutting point rectangular region coordinate information;

[0102] Step 7: According to the cutting point fine positioning coordinates and depth information of the arc plate, mapping is performed to obtain the point information in the point cloud.

[0103] Step 1: Analyze each row of pixels in the cutting point rectangular region and find the maximum value points, which represent the edges of the arc plate. Using these maximum value points, the expression of the horizontal straight line can be obtained by fitting or interpolation. Step 2: Analyze each column of pixels in the cutting point rectangular region and extract the maximum value points. Using these maximum value points, the expression of the vertical straight line can be obtained by fitting or interpolation. Step 3: Combine the expressions of the horizontal and vertical straight lines to solve for the coordinates of the intersection points. These coordinates represent the position of the arc plate and provide rough positioning coordinates of the cutting point. Step 4: Use the rough positioning coordinates of the cutting point obtained in the previous step to create a full zero mask with the same size as the cutting point rectangular region. Cover the mask on the cutting point rectangular region image and set the pixel values of the uncovered area to zero. Repeat steps 1 to 3 to obtain the position of the cutting point in the masked image. Step 5: Center the mask positioning coordinates and traverse the surrounding area to select the required pixel points as the precise positioning coordinates of the cutting point according to the pre-set conditions. If no pixel point meets the conditions, use the initial obtained coordinates as the precise positioning coordinates of the cutting point. Step 6: Associate or correct the precise positioning coordinates of the mask cutting point with the coordinate information of the cutting point rectangular region. Further determine the precise position of the arc plate cutting point and obtain the depth information. Step 7: Use the precise positioning coordinates and depth information of the arc plate cutting point to perform spatial mapping. Map the precise positioning coordinates of the arc plate cutting point to the point cloud data to obtain the point information of the arc plate in the point cloud.

[0104] The present embodiment can obtain the precise positioning coordinates of the arc plate cutting point through multiple rounds of processing and optimization of the image, making subsequent analysis and processing more accurate and reliable. By obtaining the expressions of the horizontal and vertical straight lines in the cutting point rectangular region, the edge and feature information of the arc plate can be obtained, which is helpful for subsequent point cloud generation and analysis. By creating a full zero mask and iterative processing, the positioning range of the cutting point can be continuously reduced, improving the accuracy of point positioning. By mapping the information of the arc plate cutting point to the point cloud, the three-dimensional point information of the arc plate can be obtained.

[0105] In the seventh embodiment, the method for detecting the cutting point of the arc plate on the end of the steel pipe further comprises:

[0106] Extracting the maximum value points and the number of maximum value points of each row of pixels in the cutting point rectangular region of the arc plate, and retaining the maximum value points if the number of maximum value points is less than 3; and determining the coordinate data according to the maximum value points.

[0107] Filtering the coordinate points of the coordinate data to obtain filtered coordinate data.

[0108] Performing least square fitting processing on the filtered coordinate data to obtain a horizontal straight line expression.

[0109] Embodiment eight, a steel pipe end arc extinguishing plate cutting point detection system is provided, the system comprises:

[0110] A data acquisition module is configured to acquire point cloud data of the arc extinguishing plate.

[0111] A point cloud data cropping module is configured to crop the point cloud data and process the cropped point cloud data by using a statistical method to obtain purified point cloud data.

[0112] A projection module is configured to project the purified point cloud data according to an extrinsic parameter matrix to obtain a projected two-dimensional image and a depth image.

[0113] A preprocessing module is configured to preprocess the projected two-dimensional image to obtain an arc extinguishing plate contour.

[0114] A cutting point rectangular region acquisition module is configured to establish a deep learning model and detect a cutting point rectangular region of the arc extinguishing plate by using the deep learning model.

[0115] A pipe end arc extinguishing plate cutting point acquisition module is configured to determine a pipe end arc extinguishing plate cutting point according to fitting interpolation and the cutting point rectangular region of the arc extinguishing plate.

[0116] Embodiment nine, a computer readable storage medium is provided, the computer readable storage medium is configured to store a computer program, and the computer program is configured to execute the steel pipe end arc extinguishing plate cutting point detection method according to any one of the embodiments one to seven.

[0117] Embodiment ten, a computer device is provided, the computer device comprises a memory and a processor, and the memory is configured to store a computer program, and the processor is configured to execute the steel pipe end arc extinguishing plate cutting point detection method according to any one of the embodiments one to seven when the processor executes the computer program stored in the memory.

[0118] Embodiment eleven, referring to Figure 4 and Figure 5 The embodiment is described. The embodiment is a specific embodiment of the steel pipe end arc extinguishing plate cutting point detection method according to the embodiment one, and is also configured to explain the embodiments one to seven, and specifically:

[0119] A steel pipe end arc extinguishing plate cutting point detection method based on deep learning has the following specific steps:

[0120] Step 1, arc extinguishing plate point cloud data acquisition and processing:

[0121] (1) Point cloud acquisition. A 3D camera is installed on the flange of a six-axis robot, and the camera extrinsic matrix is determined by a robot hand-eye calibration method. When acquiring point cloud data, the 3D camera is located between 600 and 800 mm above the workpiece to be detected. The 3D camera is triggered by TCP to take pictures, and the acquired point cloud data is stored.

[0122] (2) Point cloud processing. First, according to the workpiece model and batch information, the acquired point cloud data is cut along the X, Y and Z directions, and then the noise and outliers in the point cloud are filtered by statistical methods. Then, according to the extrinsic matrix, project it into a two-dimensional image I and a depth image D.

[0123] Step 2, two-dimensional image preprocessing:

[0124] First, the projected two-dimensional image is convolved and filtered using the median filter algorithm and the Gaussian filter algorithm. Then, the edge profile information of the image is extracted using the Canny algorithm, and the profile binary image is obtained using the Otsu method. Finally, the main profile of the image is extracted according to the eight-neighborhood of the pixel points, and the isolated noise points and profile noise are removed to obtain the image I', as shown in Figure 2 .

[0125] Step 3, deep learning model detection of cutting point rectangular region:

[0126] The cutting point rectangular region of the image I' is extracted by the target detection method in deep learning. The deep learning basic structure uses YOLOv5s, which is divided into four parts: input end, backbone network, neck network and output end (Prediction). The backbone network and neck network structure are mainly composed of CBL, FOCUS, CSP, SPP and Conv modules. Since the image I is an 8-bit single-channel binary image, and the image content is simple, the detection category is only four, in order to reduce the calculation complexity, improve the detection efficiency and detection accuracy, the original structure is modified, the specific modifications are as follows:

[0127] Input end modification, remove Mosaic data enhancement method, use image random rotation, flip and translation to enhance the data, and modify the input image shape to (320, 320, 1);

[0128] Modification of the Neck Network and Output: Since the scale of the artifact in image I′ is relatively stable and the scale of the detected objects in the image varies little, and the scale of the target area is large, the effectiveness of small-scale feature layers is reduced. Therefore, based on the original YOLOv5s, this method removes the 80×80 feature layer from the output structure and the associated CBL, CSP, and Concat modules in the Neck Network. The 40×40 and 20×20 feature layers and their associated Neck Network modules are retained, reducing the model's structural complexity and further improving recognition speed. Due to the change in input image shape, the original 80×80 feature layer in the output becomes a 40×40 feature layer, and the 40×40 feature layer becomes a 20×20 feature layer, while the backbone network remains unchanged. Therefore, only the 20×20 feature layer in the output and the associated CBL, CSP, Concat, and Conv modules in the Neck Network need to be removed. The modified network structure is as follows Figure 3 shown.

[0129] The coordinate information of the rectangular area of ​​the left and right cutting points is obtained by detecting the modified model, and the rectangular area images of the cutting points on the left and right sides of the image D are cropped according to the proportion, and are recorded as D l and D r ,like Figure 4 and Figure 5 shown.

[0130] Step 4: Determine the cutting point:

[0131] Since the rectangular area images of the cutting points on the left and right sides are similar, the right image D r After horizontal flipping, it has the same image as the left image D l The same features, therefore, only the left image D l Perform analysis and calculation, assuming D l The width and height are (w,h).

[0132] (1) Rough positioning of cutting points

[0133] Extract the maximum point and the number of maximum points of each row of pixels. If the number of maximum points is less than 3, retain them and get the coordinate data R = {(x1, y1), (x2, y2), ..., (x n ,y n ), where n≤h;

[0134] Filter the coordinate points in the coordinate data R and retain them if they meet the following conditions;

[0135]

[0136] Obtain filtered coordinate data R'

[0137] Fit the coordinate data R' by least square method to obtain the linear expression:

[0138]

[0139] Extract the maximum value point and the number of maximum value points of each column of pixels, repeat the above steps (1)-(3) to obtain coordinate data C' and linear expression:

[0140]

[0141] Let the intersection coordinates be P(x p ,y p ), the intersection coordinates can be obtained by the following formula.

[0142]

[0143]

[0144] According to the above steps 1 to 5, the cutting point position coarse positioning coordinates of the images D l and D r are obtained, which are denoted as P l (x l ,y l ) and P r (x r ,y r ) respectively.

[0145] (2) Cutting point position fine positioning

[0146] Make a full zero mask through the cutting point position coarse positioning coordinates P l (x l ,y l ), and cover it to the image D l to obtain the image D lm , repeat the steps in 1) to obtain the positioning coordinates P l′ (x l′ ,y l′ );

[0147] Take P l′ (x l′ ,y l′ ) as the center, loop through the pixel points in x l′-5 to x l′+5 , y l′-5 to y l′+5 , and determine the cutting point position fine positioning coordinates P l′ (x l′,y l′ )

[0148] 1.|P l′ (x l′ ,y l′-1 )-P l′ (x l′ ,y l′ )|≤1,|P l′ (x l′ ,y l′-2 )-P l′ (x l′ ,y l′ )|≤1,|P l′ (x l′ ,y l′+1 )-p l′ (x l′ ,y l′ )|≥2;

[0149] 2.P l′ (x l′-1 ,y l′ )=0,|P l′ (x l′+1 ,y l′ )- l′ (x l′-1 ,y l′ )|≠0,|P l′ (x l′+2 ,y l′ )-P l′ (x l′-1 ,y l′ )|≠0;

[0150] 3.|P l′ (x l′ ,y l′-1 )-P l′ (x l′ ,y l′ )|≤2,|P l′ (x l′ ,y l′-2 )-P l′ (x l′ ,y l′ )|≤2,|P l′ (x l′ ,y l′+1 )-P l′ (x l′ ,y l′ )|≥3;

[0151] 4.P l′ (x l′-1 ,y l′ )≠0,|Pl′ (x l′-1 ,y l′ )-P l′ (x l′ ,y l′ )|≥2,|P l′ (x l′+1 ,y l′ )-P l′ (x l′ ,y l′ )|≤1,|p l′ (x l′+2 ,y l′ )-P l′ (x l′ ,y l′ )|≤1;

[0152] If P l′ (x l′ ,y l′ ) If any one of the conditions 1 and 3 and any one of the conditions 2 and 4 are met, the loop will be exited and the precise positioning coordinates of the cutting point will be P l′ (x l′ ,y l′ );

[0153] If there is no coordinate point that meets the conditions in (3), re-traverse, P l′ (x l′ ,y l′ ) If any one of the conditions 1, 2, 3, or 4 is met, the loop will be exited and the precise positioning coordinates of the cutting point will be P l′ (x l′ ,y l′ );

[0154] If there is no coordinate point that meets the conditions in (4), the precise positioning coordinates of the cutting point are the P initially obtained in (2). l′ (x l′ ,y l′ );

[0155] Repeat the above steps (1) to (5) to obtain image D r Upper cutting point precise positioning coordinates P r′ (x r′ ,y r′ );

[0156] Set the cutting point to P l′ (x l′ ,y l′ ) and P r′ (x r′ ,y r′According to the obtained left and right cutting point rectangular region coordinate information conversion to the coordinates in the image D, and according to the coordinate information and the depth information, the point cloud is mapped into the point position information.

[0157] The present embodiment also collects 2986 point cloud data (positive samples) of the steel pipe end with the leading and arc extinguishing plate in different models, different postures and different production environments, randomly selects 2090 positive samples as the training data set, 896 positive samples as the data set, and simultaneously selects 104 point cloud data of the steel pipe end without the leading and arc extinguishing plate as the negative sample test data set. The above data sets are used to train Faster-RCNN, YOLOv3, YOLOv3-Tiny, YOLOv5s and the modified model described in the present embodiment on the same device, and the iteration is 300 times. The four network models trained using the above network structure and the modified network model described in the present embodiment are tested on the test data set, the positive sample test set is tested, as shown in Table 1, and the negative sample data set result is shown in Table 2.

[0158] Table 1 Positive sample test results

[0159]

[0160] Table 2 Negative sample test results

[0161]

[0162]

[0163] As can be seen from Table 1, the modified network model described in the present embodiment has a faster detection speed while having a higher detection accuracy, a lower missed detection rate and a lower false detection rate in the edge area detection of the steel pipe end and the leading and arc extinguishing plate. As can be seen from Table 2, the modified network model described in the present embodiment also has a lower false detection rate in negative sample detection.

[0164] Due to the requirements of the field environment and the actual production situation, the camera is installed on the robot tool, and the point cloud data is collected at a fixed point each time. The TCP precision of the field robot is ±0.2mm through six-point teaching, which has a certain error, so the cutting point space precision is tested for comprehensive accuracy. Through the laser point positioning technology, laser tracking technology and the method of the present embodiment, 147 cutting point positions of the leading and arc extinguishing plate in different environments and different postures are tested. Each test compares the robot TCP tip point with the actual cutting point position, and the comprehensive error and the average detection time in x, y and z directions are counted, and the results are shown in Table 3.

[0165] Table 3 Comprehensive error and average detection time

[0166]

[0167] As can be seen from Table 3, for the spatial positioning of the arc extinguishing plate cutting point under different environments and different postures, the detection rate of the method described in the embodiment can reach 100%, which is obviously better than the other two detection methods. And affected by the on-site environment and the interference of the robot movement, the pose of the arc extinguishing plate point cloud data collection is in a non-ideal pose, which further affects the spatial positioning accuracy of the arc extinguishing plate cutting point. However, the comprehensive error and average error of the method described in the embodiment in the x, y and z directions are both less than ±2 mm, which all meet the actual production requirements. At the same time, compared with the laser point positioning technology and the laser tracking technology, the method described in the embodiment greatly improves the detection speed.

[0168] Although the preferred embodiments of the present disclosure have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including all changes and modifications falling within the scope of the present disclosure. Obviously, those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

[0169] Those skilled in the art will appreciate that embodiments of the present disclosure can be provided as methods, systems or computer program products. Therefore, the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one flow or multiple flows and / or blocks Figure 1apparatuses that implement the functions specified in the flowchart or flowcharts and / or blocks Figure 1 flowchart or flowcharts and / or blocks Figure 1 flowchart or flowcharts and / or blocks Figure 1 flowchart or flowcharts and / or blocks Figure 1 flowchart or flowcharts and / or blocks

[0171] Finally, it should be noted that the above examples are merely used to illustrate the technical solutions of the present disclosure, rather than to limit the protection scope thereof. Although the present disclosure has been described in detail with reference to the above examples, it should be understood by those of ordinary skill in the art that, after reading the present disclosure, they can make various changes, modifications or equivalent replacements to the specific implementation manners of the present disclosure. However, these changes, modifications or equivalent replacements are all within the protection scope of the disclosed patent pending claims.

Claims

1. A method for detecting a cutting point position of an arc extinguishing plate at a pipe end of a steel pipe, characterized by, The method comprises: S1: collecting point cloud data of the arc extinguishing plate; S2: clipping the point cloud data, and using a statistical method to process the clipped point cloud data to obtain purified point cloud data; S3: projecting the purified point cloud data according to an external parameter matrix to obtain a projected two-dimensional image and a depth image; S4: preprocessing the projected two-dimensional image to obtain an arc extinguishing plate contour; S5: establishing a deep learning model, and detecting a cutting point position rectangular region of the arc extinguishing plate through the deep model; S6: determining a pipe end arc extinguishing plate cutting point position according to fitting interpolation and the cutting point position rectangular region of the arc extinguishing plate. The method for establishing the deep model of S5 is: Removing Mosaic data enhancement from the input end of the YOLOv5s network, and adding a new data enhancement method to the input end; Null processing is performed on the backbone network of the YOLOv5s network; The new data enhancement method is: image random rotation, flipping, translation and modification; removing feature map layers from an output of a YOLOv5s network ​ According to the removed output end, the associated CBL, CSP, Concat and Conv modules in the neck network of the YOLOv5s network are removed; A deep model is constructed according to the modified input end, the backbone network, the modified neck network and the modified output end; The step S6 comprises: Step 1: obtaining the maximum value point of each row of pixels in the cutting point position rectangular region of the arc extinguishing plate and the number of maximum value points, and obtaining a horizontal straight line expression according to the maximum value point; Step 2: extracting the maximum value point of each column of pixels in the cutting point position rectangular region of the arc extinguishing plate and the number of maximum value points, and obtaining a vertical straight line expression; Step 3: determining the intersection point coordinates and the cutting point position coarse positioning coordinates according to the horizontal straight line expression and the vertical straight line expression; Step 4: making a full zero mask according to the cutting point position coarse positioning coordinates, and covering it to the cutting point position rectangular region image, repeating steps 1 to 3 to obtain mask positioning coordinates; Step 5: taking the mask positioning coordinates as the center, traversing the pixel points, and ending the traversal when a preset condition is met to determine the mask cutting point position fine positioning coordinates; if there is no coordinate point meeting the condition, the mask cutting point position fine positioning coordinates are the initially obtained coordinates; Step 6: obtaining the arc extinguishing plate cutting point position fine positioning coordinates and depth information according to the mask cutting point position fine positioning coordinates and the cutting point position rectangular region coordinate information; Step 7: mapping according to the arc extinguishing plate cutting point position fine positioning coordinates and the depth information to obtain point position information in the point cloud.

2. The method according to claim 1, characterized in that, The step S1 comprises: A 3D camera is installed on a six-axis robot flange, and a camera external parameter matrix is determined by a robot hand-eye calibration method; A 3D camera is triggered by communicating with the robot through TCP, and a photograph is taken in the camera external parameter matrix to obtain point cloud arc extinguishing plate data.

3. The method of claim 1, wherein the method is characterized by: The step S4 comprises: Convolution filtering processing is performed on the projected two-dimensional image by a median filtering algorithm and a Gaussian filtering algorithm; Edge contour information of the two-dimensional image after filtering is extracted by using a Canny algorithm; A binary image of the edge contour information is obtained by Otsu method. According to the eight-neighborhood of the pixel points, the main body contour of the binary image is extracted, and isolated noise points and contour noise are removed to obtain the arc plate profile.

4. The method of claim 1, wherein the method is characterized by: The modifying is modifying a size of the input image to .

5. The method of claim 1, wherein the method is characterized by: The obtaining of the transverse straight line expression comprises: The maximum value points and the number of maximum value points of each row of pixels in the arc plate cutting point rectangular region are extracted, and if the number of maximum value points is less than 3, the maximum value points are retained, and the coordinate data is determined according to the maximum value points; The coordinate points of the coordinate data are filtered to obtain filtered coordinate data; The filtered coordinate data is subjected to least square fitting processing to obtain a transverse straight line expression.

6. A steel pipe end arc striking and extinguishing plate cutting point detection system, characterized in that: The system is realized based on the point position detection method of claim 1, and the system comprises: A data acquisition module is configured to acquire point cloud data of the arc plate; A point cloud data cropping module is configured to crop the point cloud data and process the cropped point cloud data by using a statistical method to obtain purified point cloud data; A projection module is configured to project the purified point cloud data according to an external parameter matrix to obtain a projected two-dimensional image and a depth image; A preprocessing module is configured to preprocess the projected two-dimensional image to obtain an arc plate profile; A cutting point rectangular region acquisition module is configured to establish a deep learning model and detect the cutting point rectangular region of the arc plate by using the deep learning model; A pipe end arc plate cutting point position acquisition module is configured to determine the pipe end arc plate cutting point position according to fitting interpolation and the cutting point rectangular region of the arc plate; The method for establishing the deep learning model comprises: Mosaic data enhancement is removed from the input end of the YOLOv5s network, and a new data enhancement method is added to the input end; The backbone network of the YOLOv5s network is nullified; The new data enhancement method comprises random rotation, flipping, translation and modification of an image; removing feature map layers from an output of a YOLOv5s network ​ The associated CBL, CSP, Concat and Conv modules in the neck network of the YOLOv5s network are removed according to the removed output end; The deep learning model is constructed according to the modified input end, the backbone network, the modified neck network and the modified output end.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store a computer program, and the computer program executes the steel pipe end arc plate cutting point position detection method of any one of claims 1-5.

8. A computer device, comprising: The computer readable storage medium is configured to store a computer program, and the computer program executes the steel pipe end arc plate cutting point position detection method of any one of claims 1-5.

Images