Plug anomaly detection method and system

Abstract

The invention provides a plug abnormity detection method and a detection system, which are used for detecting falling and surface flaws of a plug during operation. The method comprises the following steps: firstly, preprocessing original point cloud data to obtain preprocessed point cloud data, and segmenting the preprocessed point cloud data to obtain header data; and then plug falling detection is carried out, if the plug does not fall, plug surface flaw detection is carried out, and if the plug falls, the steps of data preprocessing and plug falling detection are repeated.

Description

Technical Field

[0001] The present invention relates to the field of industrial detection technology, and in particular to a method and system for detecting abnormality of a plug. Background Art

[0002] Seamless steel pipes have high strength and good corrosion resistance and are widely used in industrial fields such as petroleum, natural gas, and chemicals. PQF limited mandrel continuous rolling tube technology has been widely used in the global seamless steel pipe production due to its advantages of high efficiency, high yield rate, and excellent steel pipe quality. It mainly includes heating, perforation, leveling, diameter reduction, cooling, and straightening. Among them, perforation is the key process. The steel billet is formed in the center through the rotation of the roller and the push of the perforating head in the perforator. A hole. In daily production, the head may fall off due to high temperature and mechanical impact, wear of the connection between the head and the ejector, failure of the locking or ejector device, improper operation, etc. If the head falls and is not discovered in time, the ejector directly contacts the steel billet, which is very likely to cause damage to the ejector and steel roller equipment or even scrapping, increasing equipment maintenance costs and production stoppage costs.

[0003] In addition, the plug is an important component that withstands extreme working environments during the piercing process. It is in a working environment of high temperature, high pressure, strong friction, and alternating rapid cooling and heating for a long time. These harsh conditions will cause local wear of the plug surface material, crack expansion, and even coating peeling. With the accumulation of working time, these small damages gradually expand and form more serious surface defects. The surface defects of the plug have a direct impact on the quality of the steel pipe during the piercing process. During the piercing operation, damage to the plug surface may form defects such as scratches, pits or cracks inside the steel pipe, resulting in damage to the inner wall of the steel pipe. In severe cases, the entire steel pipe may even become unqualified. Not only does it reduce the yield rate, but it may also have an adverse effect on the mechanical properties and service life of downstream products.

[0004] In the current production process, whether the plug falls or not and whether there are defects on the plug surface mainly rely on manual observation. This method requires workers to concentrate for a long time, which can easily cause visual fatigue. In addition, the plug runs fast and it is difficult to judge its status by naked eyes. During the hot rolling process, the borax and water vapor raised on site will interfere with the detection effect. With the development of optical systems and automation technology, non-contact plug automatic detection methods based on machine vision have become a development trend. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the optical plug detection system in the prior art is easily interfered by impurities and has low reliability and convenience.

[0006] The present invention solves the above technical problems through the following technical means:

[0007] The method for detecting abnormal top head includes the following steps:

[0008] S1, preprocessing the original point cloud data to obtain preprocessed point cloud data, and segmenting it to obtain vertex data;

[0009] S2, plug drop detection, if the plug does not drop, execute step S3, if the plug drops, repeat steps S1 and S2; the plug drop detection includes the following steps:

[0010] S21, performing lateral projection on the plug data to reduce the data dimension and obtain a lateral projection image of the plug;

[0011] S22, annotating data of the lateral projection image of the top head and building a deep learning detection model;

[0012] S23, reasoning the lateral projection image of the mandrel based on the deep learning detection model, determining the mandrel status, and obtaining drop determination information;

[0013] S24, performing a drop alarm according to the drop determination information;

[0014] S3, plug surface defect detection, including the following steps:

[0015] S31, performing ray projection on the plug data, projecting the plug surface as a depth image with the plug surface as a reference plane, and obtaining a projection image of the plug cross-section contour;

[0016] S32, binarizing the depth image according to the depth threshold to generate a binary image;

[0017] S33, extract the connected area of ​​the binary image, and determine whether the area meets the conditions. If so, perform a retain operation and alarm. If not, perform a discard operation.

[0018] Furthermore, the specific process of step S1 is as follows:

[0019] S11, collecting and obtaining original point cloud data;

[0020] S12, downsampling the original point cloud data through voxel filtering to obtain a downsampled point cloud;

[0021] S13, density-based point cloud segmentation, based on the density of points within the domain, points with accessible density are classified into the same cluster, points with unreachable density are divided into new clusters, and all points are traversed until all points have corresponding clusters, thus obtaining segmented point clouds and segmented clusters;

[0022] S14, introduce preset conditions for the segmented point cloud, screen the segmented clusters according to the preset conditions, and obtain the target cluster, wherein for each cluster C i , calculate the coordinates of the center point (xc ,y c ,z c );

[0023] S15. Determine cluster C i Whether the preset point condition and the preset center condition are met; i The coordinates of the center point (x c ,y c ,z c ), compared with the preset coordinate range;

[0024] S16. If yes, then for the current cluster C i Execute the retention operation;

[0025] S17: If not, a discard operation is performed on the current cluster.

[0026] Further, the S21 includes:

[0027] S211, input point cloud data;

[0028] S212, calculating the projection point cloud coordinate range according to the point cloud data, wherein the boundaries of the y coordinate and the z coordinate in the point cloud data are calculated, the physical boundaries of the two-dimensional image are defined, and the size of the two-dimensional projection image is determined:

[0029] y min =min(y i ),y max =max(y i ),z min =min(z i ),z max =max(z i );

[0031] S213, calculating the image resolution in the y direction and the z direction according to the boundary value of the physical boundary of the two-dimensional image and the size of the two-dimensional projection image:

[0032]

[0033] S214, calculating coordinate projection;

[0034] S215, acquiring a lateral projection image, and obtaining the image pixel position through a coordinate conversion operation.

[0035] Furthermore, in S215, the y coordinate and z coordinate of each point in the point cloud data are converted to the image pixel position (i, j) using the following mapping logic:

[0036]

[0037] Furthermore, in step S31, the least square method is used to perform ellipse fitting on each contour of the processed plug data to obtain the center of the ellipse (x 0 ,y 0 ) major axis a, minor axis b, rotation angle ω; plane projection is performed on the top point cloud data by using a ray projection method, and the point cloud data is projected as a depth image with its surface as the reference plane. The ray projection steps include:

[0038] S311, phase angle θ from the center to the start 1 Emit ray l 1 As the starting direction, the phase angle θ from the center to the end n Emit ray l n As the termination direction, θ 1 and θ n Multiple rays l are emitted with a fixed step length i , satisfying θ i+1 -θ i =step;

[0039] S312, calculate the phase and Euclidean distance of each point to the center of the ellipse. Specifically, for each point (x s ,y s ), the phase and Euclidean distance are calculated as

[0040] S313, calculating the contour point with the closest phase distance on each ray within a given boundary threshold range as the representative point of the ray. Specifically, for ray l i Its nearest contour point (x k ,y k )satisfy like Then the contour point is ray l i The representative point of the ray l i There is no representative point;

[0041] S314, calculate the distance from the representative point to the ellipse surface, according to the ellipse parameters and the phase of the representative point by the formula The radius of the ellipse in the direction of the representative point can be calculated, so the distance from the representative point to the ellipse surface can be obtained by Calculate, if d s >0 means the point is above the boundary. s <0 means the point is below the boundary;

[0042] S315, depth map grayscale value calculation, grayscale value can be expressed by Calculate, where RES zis the resolution in the z direction, pixelMid is the grayscale median; the width of the final generated depth image is the number of rays n, and the height is the number of contours in the point cloud data; due to the inconsistent radius of each contour in the top data, the resolution of each row of data in the depth image in the width direction is different, so it is calculated separately, and the calculation formula is

[0043] Furthermore, the binary image generation method in step S32 is: based on a given depth threshold depth th , through the formula Calculate the corresponding grayscale threshold and compare the grayscale value of the depth map with the grayscale value of the depth map. th By comparison, the formula Generate a binary image B(x,y), where G(x,y) represents the grayscale value of (x,y).

[0044] Furthermore, the method for calculating the connected regions of the binary image in step S33 is as follows: for any connected region R k , whose area is RES x is the x-direction resolution, RES y is the resolution in the y direction; if S k Satisfy the condition S k ≥S min , then the connected region is considered to be a defect.

[0045] The present invention also provides a plug abnormality detection system, comprising:

[0046] A data preprocessing module is used to preprocess the original point cloud data, obtain the preprocessed point cloud data, and segment the point cloud data to obtain the vertex data;

[0047] The plug drop detection module is used for plug drop detection. If the plug has not dropped, step S3 is executed. If the plug has dropped, steps S1 and S2 are repeated. The plug drop detection includes the following steps:

[0048] S21, performing lateral projection on the plug data to reduce the data dimension and obtain a lateral projection image of the plug;

[0049] S22, annotating data of the lateral projection image of the top head and building a deep learning detection model;

[0050] S23, reasoning the lateral projection image of the mandrel based on the deep learning detection model, determining the mandrel status, and obtaining drop determination information;

[0051] S24, performing a drop alarm according to the drop determination information;

[0052] The plug surface defect detection module is used for plug surface defect detection, and includes the following steps:

[0053] S31, performing ray projection on the plug data, projecting the plug surface as a depth image with the plug surface as a reference plane, and obtaining a projection image of the plug cross-section contour;

[0054] S32, binarizing the depth image according to the depth threshold to generate a binary image;

[0055] S33, extract the connected area of ​​the binary image, and determine whether the area meets the conditions. If so, perform a retain operation and alarm. If not, perform a discard operation.

[0056] Furthermore, the step S21 includes:

[0057] S211, input point cloud data;

[0058] S212, calculating the projection point cloud coordinate range according to the point cloud data, wherein the boundaries of the y coordinate and the z coordinate in the point cloud data are calculated, the physical boundaries of the two-dimensional image are defined, and the size of the two-dimensional projection image is determined:

[0059] y min =min(y i ),y max =max(y i ),z min =min(z i ),z max =max(z i );

[0061] S213, calculating the image resolution in the y direction and the z direction according to the boundary value of the physical boundary of the two-dimensional image and the size of the two-dimensional projection image:

[0062]

[0063] S214, calculating coordinate projection;

[0064] S215, acquiring a lateral projection image, and obtaining the image pixel position through a coordinate conversion operation.

[0065] Furthermore, in S215, the y coordinate and z coordinate of each point in the point cloud data are converted to the image pixel position (i, j) using the following mapping logic:

[0066]

[0067] Furthermore, in step S31, the least square method is used to perform ellipse fitting on each contour of the processed plug data to obtain the center of the ellipse (x 0 ,y 0 ) major axis a, minor axis b, rotation angle ω; plane projection is performed on the top point cloud data by using a ray projection method, and the point cloud data is projected as a depth image with its surface as the reference plane. The ray projection steps include:

[0068] S311, phase angle θ from the center to the start 1 Emit ray l 1 As the starting direction, the phase angle θ from the center to the end n Emit ray l n As the termination direction, θ 1 and θ n Multiple rays l are emitted with a fixed step length i , satisfying θ i+1 -θ i =step;

[0069] S312, calculate the phase and Euclidean distance of each point to the center of the ellipse. Specifically, for each point (x s ,y s ), the phase and Euclidean distance are calculated as

[0070] S313, calculating the contour point with the closest phase distance on each ray within a given boundary threshold range as the representative point of the ray. Specifically, for ray l i Its nearest contour point (x k ,y k )satisfy like Then the contour point is ray l i The representative point of the ray l i There is no representative point;

[0071] S314, calculate the distance from the representative point to the ellipse surface, and calculate the distance from the representative point to the ellipse surface according to the ellipse parameters and the phase of the representative point. By The radius of the ellipse in the direction of the representative point can be calculated, so the distance from the representative point to the ellipse surface can be obtained by Calculate, if d s >0 means the point is above the boundary. s <0 means the point is below the boundary;

[0072] S315, depth map grayscale value calculation, grayscale value can be expressed by Calculate, where RES zis the resolution in the z direction, pixelMid is the grayscale median; the width of the final generated depth image is the number of rays n, and the height is the number of contours in the point cloud data; due to the inconsistent radius of each contour in the top data, the resolution of each row of data in the depth image in the width direction is different, so it is calculated separately, and the calculation formula is

[0073] Furthermore, the binary image generation method in step S32 is: based on a given depth threshold depth th , through the formula Calculate the corresponding grayscale threshold and compare the grayscale value of the depth map with the grayscale value of the depth map. th By comparison, the formula Generate a binary image B(x,y), where G(x,y) represents the grayscale value of (x,y).

[0074] Furthermore, the method for calculating the connected regions of the binary image in step S33 is as follows: for any connected region R k , whose area is RES x is the x-direction resolution, RES y is the resolution in the y direction; if S k Satisfy the condition S k ≥S min , then the connected region is considered to be a defect.

[0075] The advantages of the present invention are:

[0076] The present invention projects the processed point cloud data into a 2D image by lateral projection, and detects the 2D projection image based on the deep learning pattern recognition method to determine the state of the plug. This method not only improves the reliability and convenience of plug detection, but also provides new ideas for subsequent tasks such as plug melt tip detection and plug meat loss detection, and has a reference role in research in related fields.

[0077] Since the plug has obvious longitudinal (y direction) and height (z direction) features, the yOz plane can well show these features and clearly display the outline and size of the plug. However, the data in the x direction is relatively redundant and contributes less to the judgment of the plug state. Selecting the yOz plane not only retains the main shape features of the plug, but also makes the features after data projection more intuitive, providing a basis for subsequent morphological recognition.

[0078] The present invention uses ray projection to convert the preprocessed point cloud into a depth image, and then applies image binarization and connected region analysis technology to screen out defect areas that meet the depth threshold and area requirements. This method not only improves the accuracy and efficiency of plug defect detection, but also provides a reference for further research in related fields.

[0079] In the point cloud filtering and point cloud segmentation operations of the present invention, the main purpose of filtering is to downsample the original point cloud, remove redundant data while ensuring data semantics and detail information, and reduce the amount of calculation of the entire processing flow. Point cloud segmentation is to distinguish the top data from the noise data caused by external factors such as water vapor and borax, so as to improve data quality.

[0080] The density-based point cloud segmentation of the present invention can segment the point cloud into different clusters according to the connectivity between the point clouds, thereby improving the segmentation effect of block noise points and reducing false detection or missed detection caused by a large amount of noise data. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 The figure is a flowchart of the overall process of detecting abnormal plug in an embodiment of the present invention.

[0082] Figure 2 Schematic diagram of basic steps of a method for detecting a plug drop based on lateral projection in an embodiment of the present invention;

[0083] Figure 3 Schematic diagram of specific steps of preprocessing the original point cloud in an embodiment of the present invention;

[0084] Figure 4 Schematic diagram of specific steps of performing lateral projection in an embodiment of the present invention;

[0085] Figure 5 A schematic diagram of basic steps of a plug surface defect detection method based on ray projection in an embodiment of the present invention;

[0086] Figure 6 It is a schematic diagram of ray projection in an embodiment of the present invention;

[0087] Figure 7 Figure 8 This is a test diagram using the method of the embodiment of the present invention. Figure 7 The upper left picture in the middle is the image taken by the 3D camera, and the lower left picture is the image after lateral projection. Figure 8 The results of the tests at different time points are shown. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0089] This embodiment provides a plug abnormality detection method for detecting plug drop and surface defects during operation. Figure 1 As shown, the following steps are included:

[0090] Step S1, preprocessing the original point cloud data to obtain preprocessed point cloud data, and segmenting it to obtain vertex data;

[0091] Step S2, plug drop detection, if the plug does not drop, execute step S3, if the plug drops, repeat steps S1 and S2; the plug drop detection includes the following steps:

[0092] S21, performing lateral projection on the plug data to reduce the data dimension and obtain a lateral projection image of the plug;

[0093] S22, annotating data of the lateral projection image of the top head and building a deep learning detection model;

[0094] S23, reasoning the lateral projection image of the mandrel based on the deep learning detection model, determining the mandrel status, and obtaining drop determination information;

[0095] S24, performing a drop alarm according to the drop determination information;

[0096] Step S3, plug surface defect detection, includes the following steps:

[0097] S31, performing ray projection on the plug data, projecting the plug surface as a depth image with the plug surface as a reference plane, and obtaining a projection image of the plug cross-section contour;

[0098] S32, binarizing the depth image according to the depth threshold to generate a binary image;

[0099] S33, extract the connected area of ​​the binary image, and determine whether the area meets the conditions. If so, perform a retain operation and alarm. If not, perform a discard operation.

[0100] The following describes data preprocessing, plug drop detection, and plug surface defect detection in detail.

[0101] 1. Data Preprocessing

[0102] like Figure 2 As shown, in this embodiment, the point cloud data is collected by the 3D profiler arranged at the top working position. In the working environment, a cooling device has been arranged to cool the top working position environment to reduce the influence of water vapor on the collected data, but there is still a small amount of water vapor and other influencing factors in the environment, resulting in large noise in the point cloud data.

[0103] Voxel filtering downsampling;

[0104] In this embodiment, there are a large number of dense points in the original 3D point cloud data, which contains redundant information. Downsampling the point cloud on the basis of meeting the task accuracy requirements is a prerequisite for ensuring the real-time performance of the algorithm. Voxel filtering is a commonly used downsampling method. Its core idea is to divide the point cloud into 3D grids (i.e., voxels), and replace all the points in the voxel with the representative point in each voxel, so as to reduce the quantity of the point cloud data. For the input point cloud, its boundary range is [X min , X max , [Y min , Y max , [Z min , Z max . If the side lengths of the voxel grid are defined as (l x , l y , l z ), then for each point (x i , y i , z i ), the calculation method of the index of the voxel where it is located is:

[0105]

[0106] For each voxel grid, the geometric center of all the points it contains is used as the representative point of this voxel, and the calculation method is:

[0107]

[0108] Obtain the main cluster based on density segmentation;

[0109] In this embodiment, density-based point cloud segmentation can divide the point cloud into different clusters according to the connectivity state between points, so as to improve the segmentation effect on成片的块状噪点 (成片的块状噪点 can be translated as "成片的块状噪声点" in English, but it seems there may be an incorrect expression here, you can check and correct it if necessary), and reduce false detection or missed detection caused by a large amount of noise data. First, set the status of all points to unvisited. Then, for each point p, if the point has not been visited, mark the point as visited. And calculate the number of points N ε (p) within the neighborhood range ε of this point, which can be expressed as:

[0110] N ε (p) = {q ∈ D | dist(p, q) ≤ ε}

[0111] where D is the point cloud data set, and dist(p, q) represents the distance between p and q. For the preset point number threshold minPts, if N ε (p) < minPts, mark this point as a noise point. If N ε (p) ≥ minPts, mark this point as a core point, and create a new cluster, and add p and all the points within its neighborhood to this cluster. Then expand this cluster. For each point q in the cluster, calculate the ε neighborhood N ε(q), if N ε (q)>minPts, then all unvisited points in the neighborhood of q are added to the current cluster. In this way, each density-reachable point is traversed step by step until no new core point can be found. Repeat the above steps until all points are visited. For the segmented point cloud, there may be some small clusters that do not meet the target requirements or noise clusters that deviate from the specified area. In order to further extract the target clusters that meet the requirements, the following conditions are introduced to screen the segmented clusters to ensure that the retained clusters meet the specific spatial range and density requirements. For each cluster C i , calculate the coordinates of its center point (x c ,y c ,z c ) and compare it with the set coordinate range. If the cluster center is not within the predefined range, the cluster is discarded. The specific calculation is as follows:

[0112]

[0113] In this embodiment, the xyz range of the cluster center is set to [x min ,x max ],[x min ,x max ],[x min ,x max ]. If x min ≤x c ≤x max And y min ≤y c ≤y max And z min ≤z c ≤z max , then the cluster center meets the conditions and proceeds to the next step. Otherwise, the cluster is discarded. To exclude small clusters caused by noise or sparse point cloud areas, clusters with points greater than a certain value are retained. Set a minimum point threshold N min , if cluster C i The points satisfy:

[0114] |C i |≥N min

[0115] If the cluster is not found, the cluster is retained; otherwise, the cluster is discarded.

[0116] 2. Head drop detection

[0117] Performing lateral projection on the plug data segmented in the aforementioned step S1 to reduce the data dimension and obtain a lateral projection image of the plug;

[0118] like Figure 4As shown, in this embodiment, the step S2 of performing the lateral projection further includes the following specific steps:

[0119] S21, input point cloud data;

[0120] In this embodiment, due to the large amount of 3D point cloud data and high dimensionality, direct processing often requires a lot of computing resources and time. Projection is an effective means to simplify calculations. Projecting point cloud data onto a two-dimensional plane can significantly reduce the amount of data, making it easier to complete subsequent analysis operations quickly and at low cost. Although some spatial information is discarded through projection, this processing method can significantly shorten the calculation time while ensuring key features.

[0121] S22, calculating the coordinate range of the projected point cloud;

[0122] In this embodiment, the boundaries of the y coordinates and z coordinates in the point cloud data are calculated. The boundary values ​​define the physical boundaries of the two-dimensional image so as to determine the size and resolution of the image. The specific calculation formula is as follows:

[0123] y min =min(y i ),y max =max(y i ),z min =min(z i ),z max =max(z i );

[0125] S23, calculating image resolution;

[0126] In this embodiment, the resolutions in the y and z directions are calculated according to the boundary value and the sizes W and H of the two-dimensional projection image:

[0127]

[0128] S24, calculating coordinate projection;

[0129] S25, acquiring a lateral projection image;

[0130] In this embodiment, the y and z coordinates of each point in the point cloud are converted to the pixel position (i, j) in the image. The mapping formula is as follows:

[0131]

[0132] S26, annotating the projected image data and building a deep learning detection model;

[0133] S27, reasoning the projection image based on the deep learning detection model, judging the state of the head, and determining whether the head falls;

[0134] S28. Activate a drop alarm.

[0135] In this embodiment, the head feature recognition model of the present invention is implemented based on a convolutional neural network. The equipment continuously collects and enriches intact and dropped images of the head under different working conditions, and gradually expands the training data set, thereby steadily improving the system's recognition accuracy and detection accuracy of the head features. The system adopts an online detection method, which effectively combines deep learning with machine vision functions: specifically, the machine vision module is responsible for image capture and processing, and obtains image information in the production process in real time; the deep learning module uses the GPU of the same host to analyze and classify image features and distinguish the head status. While ensuring the detection effect, this design greatly improves the overall computing efficiency and real-time response speed of the system, further ensuring the rapidity and reliability of production detection.

[0136] In summary, the present invention is based on the point cloud data collected by a 3D camera, and realizes data downsampling and segmentation of top data and water vapor noise data through point cloud filtering and a density-based clustering algorithm, which makes full use of the spatial three-dimensional information of the point cloud and improves the denoising effect.

[0137] In the point cloud filtering and point cloud segmentation operations of the present invention, the main purpose of filtering is to downsample the original point cloud, remove redundant data while ensuring data semantics and detail information, and reduce the amount of calculation of the entire processing flow. Point cloud segmentation is to distinguish the top data from the noise data caused by external factors such as water vapor and borax, so as to improve data quality.

[0138] The density-based point cloud segmentation of the present invention can segment the point cloud into different clusters according to the connectivity between the point clouds, thereby improving the segmentation effect of block noise points and reducing false detection or missed detection caused by a large amount of noise data.

[0139] The present invention projects the processed point cloud data into a 2D image by lateral projection, and detects the 2D projection image based on the deep learning pattern recognition method to determine the state of the plug. This method not only improves the reliability and convenience of plug detection, but also provides new ideas for subsequent tasks such as plug melt tip detection and plug meat loss detection, and has a reference role in research in related fields.

[0140] Since the plug has obvious longitudinal (y direction, i.e., plug length direction) and height (z direction) features, the yOz plane can well represent these features and clearly show the profile and size of the plug. However, the data in the x direction is relatively redundant and contributes less to the judgment of the plug state. Selecting the yOz plane not only retains the main shape features of the plug, but also makes the features after data projection more intuitive, providing a basis for subsequent morphological recognition.

[0141] like Figure 7 Figure 8 As shown, Figure 7 Figure 8 For the test image, Figure 7 The upper left picture in the middle is the image taken by the 3D camera, and the lower left picture is the image after lateral projection. From the lower left picture, it can be seen that there is a top head, which means that the top head has not fallen off. Figure 8 The display shows the detection results at different time points. Since the 3D camera shoots the top from top to bottom, the data of the lower half of the top is missing, so the image of the top after lateral projection is in a downward state. It should be noted here that Figure 7 , Figure 8 The time in the figure is the testing time, not the technology disclosure time.

[0142] 3. Plug surface defect detection

[0143] When the detection head does not fall off, continue to detect whether there are defects on the surface of the head, which specifically includes the following steps:

[0144] (1) Calculate the ray projection depth image;

[0145] In this embodiment, for the processed point cloud data, the least square method is used to perform ellipse fitting on each contour of the head data to obtain the center of the ellipse (x 0 ,y 0 ) major axis a, minor axis b and rotation angle ω. In order to ensure that the relative position of the point cloud before and after projection between the contours remains unchanged, a ray projection-based method is used to perform plane projection on the head point cloud data. The point cloud data is projected as a depth image with its surface as the reference plane for defect detection. For a schematic diagram of the projection of a single contour, see Figure 6 .

[0146] In the projection operation of this embodiment, the phase angle θ from the center of the circle to the starting point is 1 Emit ray l 1 As the starting direction, the phase angle θ from the center to the end n Emit ray l n As the termination direction, θ 1 and θ n Multiple rays l are emitted with a fixed step length i , satisfying θ i+1 -θ i =step;

[0147] like Figure 5 As shown, the phase and Euclidean distance of each point to the ellipse center are calculated. Specifically, for each point (x s ,y s ), the phase and Euclidean distance are calculated as

[0148] The contour point with the closest phase distance on each ray is calculated within a given boundary threshold range as the representative point of the ray. Specifically, for ray l i Its nearest contour point (x k ,y k )satisfy like Then the contour point is ray l i The representative point of the ray l i There is no representative point;

[0149] Calculate the distance from the representative point to the ellipse surface, based on the ellipse parameters and the phase of the representative point By The radius of the ellipse in the direction of the representative point can be calculated, so the distance from the representative point to the ellipse surface can be obtained by Calculate, if d s >0 means the point is above the boundary. s <0 means the point is below the boundary;

[0150] Depth map grayscale value calculation: The grayscale value in the depth map reflects the fluctuation of the contour surface compared to the reference surface. The grayscale value can be calculated by the formula Calculate, where RES z is the resolution in the z direction, and pixelMid is the grayscale median. The width of the resulting depth image is the number of rays n, and the height is the number of contours in the point cloud data. Since the radii of the contours in the top data are inconsistent, the resolution of each row of data in the depth image in the width direction is different, and must be calculated separately. The calculation formula is

[0151] (2) Perform image binarization based on the depth threshold;

[0152] In this embodiment, based on a given depth threshold depth th , through the formula Calculate the corresponding grayscale threshold and compare the grayscale value of the depth map with the grayscale value of the depth map. th By comparison, the formula Generate a binary image B(x,y), where G(x,y) represents the grayscale value of (x,y).

[0153] (3) Extracting connected regions;

[0154] (4) Determine whether the area meets the conditions;

[0155] (5) If yes, execute the retention operation and issue an alarm;

[0156] (6) If not, execute the discard operation.

[0157] In this embodiment, the foreground pixels are analyzed for connected regions. Specifically, if there are foreground pixels in the eight neighborhoods of the upper, lower, left, right, and four diagonal pixels of each foreground pixel, these foreground pixels belong to a group of connected regions. All foreground pixels are traversed until all connected regions are found. For any connected region R k , whose area is RES x is the x-direction resolution, RES y is the resolution in the y direction. k Satisfy the condition S k ≥S min , then the connected region is considered to be a defect.

[0158] In summary, this method is based on the point cloud data collected by the 3D camera, combined with point cloud filtering and density-based clustering algorithm, to achieve data downsampling and effective segmentation of the head data and water vapor noise data. By utilizing the three-dimensional spatial information of the point cloud, the effect of noise removal is significantly improved. On this basis, ray projection is used to convert the preprocessed point cloud into a depth image, and then image binarization and connected region analysis technology are applied to screen out defect areas that meet the depth threshold and area requirements. This method not only improves the accuracy and efficiency of head defect detection, but also provides a reference for further research in related fields.

[0159] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting abnormal plug, characterized in that: The following steps are involved: S1, preprocessing the original point cloud data to obtain preprocessed point cloud data, and segmenting it to obtain vertex data; S2, plug drop detection, if the plug does not drop, execute step S3, if the plug drops, repeat steps S1 and S2; the plug drop detection includes the following steps: S21, performing lateral projection on the plug data to reduce the data dimension and obtain a lateral projection image of the plug; S22, annotating data of the lateral projection image of the top head and building a deep learning detection model; S23, reasoning the lateral projection image of the mandrel based on the deep learning detection model, determining the mandrel status, and obtaining drop determination information; S24, performing a drop alarm according to the drop determination information; S3, plug surface defect detection, including the following steps: S31, performing ray projection on the plug data, projecting the plug surface as a depth image with the plug surface as a reference plane, and obtaining a projection image of the plug cross-section contour; S32, binarizing the depth image according to the depth threshold to generate a binary image; S33, extract the connected area of ​​the binary image, and determine whether the area meets the conditions. If so, perform a retain operation and alarm. If not, perform a discard operation.

2. The method for detecting plug abnormality according to claim 1, characterized in that: The specific process of step S1 is: S11, collecting and obtaining original point cloud data; S12, downsampling the original point cloud data through voxel filtering to obtain a downsampled point cloud; S13, density-based point cloud segmentation, based on the density of points within the domain, points with accessible density are classified into the same cluster, points with unreachable density are divided into new clusters, and all points are traversed until all points have corresponding clusters, thus obtaining segmented point clouds and segmented clusters; S14, introduce preset conditions for the segmented point cloud, filter the segmented clusters according to the preset conditions, and obtain the target cluster, where for each cluster C i , calculate the coordinates of the center point (x c ,y c ,z c ); S15. Determine cluster C i Whether the preset point condition and the preset center condition are met; i The coordinates of the center point (x c ,y c ,z c ), compared with the preset coordinate range; S16. If yes, then for the current cluster C i Execute the retention operation; S17: If not, a discard operation is performed on the current cluster.

3. The method for detecting abnormal plug head according to claim 1, characterized in that: The S21 includes: S211, input point cloud data; S212, calculating the projection point cloud coordinate range according to the point cloud data, wherein the boundaries of the y coordinate and the z coordinate in the point cloud data are calculated, the physical boundaries of the two-dimensional image are defined, and the size of the two-dimensional projection image is determined: y min =min(y i ),y max =max(y i ),z min =min(z i ),z max =max(z i ); S213, calculating the image resolution in the y direction and the z direction according to the boundary value of the physical boundary of the two-dimensional image and the size of the two-dimensional projection image: S214, calculating coordinate projection; S215, acquiring a lateral projection image, and obtaining the image pixel position through a coordinate conversion operation.

4. The method for detecting plug abnormality according to claim 3, characterized in that: In S215, the y coordinate and z coordinate of each point in the point cloud data are converted to the image pixel position (i, j) using the following mapping logic:

5. The method for detecting plug abnormality according to claim 1, characterized in that: In step S31, the least square method is used to perform ellipse fitting on each contour of the processed plug data to obtain the center (x0, y0) of the ellipse, the major axis a, the minor axis b, and the rotation angle ω; a ray projection-based method is used to perform plane projection on the plug point cloud data, and the point cloud data is projected as a depth image with its surface as the reference plane. The ray projection step includes: S311, send ray l1 from the center of the circle to the starting phase angle θ1 as the starting direction, and send ray l1 from the center of the circle to the ending phase angle θ n Emit ray l n As the termination direction, θ1 and θ n Multiple rays l are emitted with a fixed step length i , satisfying θ i+1 -θ i =step; S312, calculate the phase and Euclidean distance of each point to the center of the ellipse. Specifically, for each point (x s ,y s ), the phase and Euclidean distance are calculated as S313, calculating the contour point with the closest phase distance on each ray within a given boundary threshold range as the representative point of the ray. Specifically, for ray l i Its nearest contour point (x k ,y k )satisfy like Then the contour point is ray l i The representative point of the ray l i There is no representative point; S314, calculate the distance from the representative point to the ellipse surface, and calculate the distance from the representative point to the ellipse surface according to the ellipse parameters and the phase of the representative point. By The radius of the ellipse in the direction of the representative point can be calculated, so the distance from the representative point to the ellipse surface can be obtained by Calculate, if d s >0 means the point is above the boundary. s <0 means the point is below the boundary; S315, depth map grayscale value calculation, grayscale value can be expressed by Calculate, where RES z is the resolution in the z direction, pixelMid is the grayscale median; the width of the final generated depth image is the number of rays n, and the height is the number of contours in the point cloud data; due to the inconsistent radius of each contour in the top data, the resolution of each row of data in the depth image in the width direction is different, so it is calculated separately, and the calculation formula is 6. The method for detecting plug abnormality according to claim 1, characterized in that: The binary image generation method in step S32 is: based on a given depth threshold depth th , through the formula Calculate the corresponding grayscale threshold and compare the grayscale value of the depth map with the grayscale value of the depth map. th By comparison, the formula Generate a binary image B(x,y), where G(x,y) represents the grayscale value of (x,y).

7. The method for detecting abnormal plug head according to claim 6, characterized in that: The method for calculating the connected region of the binary image in step S33 is: k , whose area is RES y , RES x is the x-direction resolution, RES y is the resolution in the y direction; if S k Satisfy the condition S k ≥S min , then the connected region is considered to be a defect.

8. The head abnormality detection system is characterized by: include: A data preprocessing module is used to preprocess the original point cloud data, obtain the preprocessed point cloud data, and segment the point cloud data to obtain the vertex data; The plug drop detection module is used for plug drop detection. If the plug has not dropped, step S3 is executed. If the plug has dropped, steps S1 and S2 are repeated. The plug drop detection includes the following steps: S21, performing lateral projection on the plug data to reduce the data dimension and obtain a lateral projection image of the plug; S22, annotating data of the lateral projection image of the top head and building a deep learning detection model; S23, reasoning the lateral projection image of the mandrel based on the deep learning detection model, determining the mandrel status, and obtaining drop determination information; S24, performing a drop alarm according to the drop determination information; The plug surface defect detection module is used for plug surface defect detection, and includes the following steps: S31, performing ray projection on the plug data, projecting the plug surface as a depth image with the plug surface as a reference plane, and obtaining a projection image of the plug cross-section contour; S32, binarizing the depth image according to the depth threshold to generate a binary image; S33, extract the connected area of ​​the binary image, and determine whether the area meets the conditions. If so, perform a retain operation and alarm. If not, perform a discard operation.

9. The plug abnormality detection system according to claim 8, characterized in that: The step S21 comprises: S211, input point cloud data; S212, calculating the projection point cloud coordinate range according to the point cloud data, wherein the boundaries of the y coordinate and the z coordinate in the point cloud data are calculated, the physical boundaries of the two-dimensional image are defined, and the size of the two-dimensional projection image is determined: y min =min(y i ),y max =max(y i ),z min =min(z i ),z max =max(z i ); S213, calculating the image resolution in the y direction and the z direction according to the boundary value of the physical boundary of the two-dimensional image and the size of the two-dimensional projection image: S214, calculating coordinate projection; S215, acquiring a lateral projection image, and obtaining the image pixel position through a coordinate conversion operation.

10. The plug abnormality detection system according to claim 9, characterized in that: In S215, the y coordinate and z coordinate of each point in the point cloud data are converted to the image pixel position (i, j) using the following mapping logic:

11. The plug abnormality detection system according to claim 8, characterized in that: In step S31, the least square method is used to perform ellipse fitting on each contour of the processed plug data to obtain the center (x0, y0) of the ellipse, the major axis a, the minor axis b, and the rotation angle ω; a ray projection-based method is used to perform plane projection on the plug point cloud data, and the point cloud data is projected as a depth image with its surface as the reference plane. The ray projection step includes: S311, send ray l1 from the center of the circle to the starting phase angle θ1 as the starting direction, and send ray l1 from the center of the circle to the ending phase angle θ n Emit ray l n As the termination direction, θ1 and θ n Multiple rays l are emitted with a fixed step length i , satisfying θ i+1 -θ i =step; S312, calculate the phase and Euclidean distance of each point to the center of the ellipse. Specifically, for each point (x s ,y s ), the phase and Euclidean distance are calculated as S313, calculating the contour point with the closest phase distance on each ray within a given boundary threshold range as the representative point of the ray. Specifically, for ray l i Its nearest contour point (x k ,y k )satisfy like Then the contour point is ray l i The representative point of the ray l i There is no representative point; S314, calculate the distance from the representative point to the ellipse surface, according to the ellipse parameters and the phase of the representative point by the formula The radius of the ellipse in the direction of the representative point can be calculated, so the distance from the representative point to the ellipse surface can be obtained by Calculate, if d s >0 means the point is above the boundary. s <0 means the point is below the boundary; S315, depth map grayscale value calculation, grayscale value can be expressed by Calculate, where RES z is the resolution in the z direction, pixelMid is the grayscale median; the width of the final generated depth image is the number of rays n, and the height is the number of contours in the point cloud data; due to the inconsistent radius of each contour in the top data, the resolution of each row of data in the depth image in the width direction is different, so it is calculated separately, and the calculation formula is 12. The plug abnormality detection system according to claim 8, characterized in that: The binary image generation method in step S32 is: based on a given depth threshold depth th , through the formula Calculate the corresponding grayscale threshold and compare the grayscale value of the depth map with the grayscale value of the depth map. th By comparison, the formula Generate a binary image B(x,y), where G(x,y) represents the grayscale value of (x,y).

13. The plug abnormality detection system according to claim 12, characterized in that: The method for calculating the connected region of the binary image in step S33 is: k , whose area is RES y , RES x is the x-direction resolution, RES y is the resolution in the y direction; if S k Satisfy the condition S k ≥S min , then the connected region is considered to be a defect.

Images