Steel pipe identification position judging and positioning method based on 3D visual technology

Abstract

The invention discloses a steel pipe identification position judging and positioning method based on the 3D vision technology, and belongs to the technical field of industrial automation and machine vision, and the method comprises the steps: carrying out the 3D scanning of the end face of a steel pipe in a certain range at the initial position of a camera, and obtaining first point cloud data; based on the first point cloud data, the inclination angle of the steel pipe end face is calculated, the camera is adjusted to the optimal shooting pose, 3D scanning is conducted on the steel pipe end face again, and second point cloud data is obtained; based on the second point cloud data, steel pipe end face defect detection is carried out; selecting an optimal identification position on the end face of the steel pipe based on a defect detection result and the specification of the to-be-identified content; and the pose of the laser marking equipment is adjusted, and laser marking operation is carried out at the optimal identification position. According to the invention, the requirements of marking tasks of multiple sizes and multiple types of steel pipes can be met, the marking efficiency of the whole industry is improved, and the accuracy and the recognition rate of information are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of industrial automation and machine vision technology, and in particular to a method for judging and locating a steel pipe marking position based on 3D vision technology. Background Art

[0002] During the steel pipe production process, the steel pipe identification often contains a lot of important information (including billet number, furnace number, specifications, etc.), which is the main basis for material tracking and product tracing.

[0003] Marking steel pipes is a key step to ensure product quality, achieve production traceability, and meet quality inspection requirements. The selection of marking positions and the accuracy of identification information are of vital importance to subsequent production, quality inspection, and traceability.

[0004] However, traditional marking position positioning methods mainly rely on manual operation or simple 2D image processing technology. These methods are often easily affected by factors such as the environment, product specifications and dimensions, and marking position quality, resulting in unstable positioning accuracy and low efficiency.

[0005] In the traditional laser marking process, the laser and the end face are not strictly perpendicular. Due to the randomness of the marking angle selection, the marking information is often uneven in depth and the clarity is affected. The non-standard laser marking angle also causes the marked information to be distorted and overlapped, which seriously affects the subsequent information recognition and tracking tasks.

[0006] In the traditional laser marking process, the marking area is not selected, and the marking method is used directly, resulting in unclear marking of defective areas, which is prone to recognition errors. The marking content does not completely match the end face size, and there may be errors such as the marking position is not enough to fully display the marking content. Summary of the invention

[0007] The present invention provides a method for judging and locating the position of a steel pipe mark based on 3D vision technology, so as to solve the technical problems of random position selection and poor marking effect in the laser marking process in the prior art.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0009] On the one hand, the present invention provides a method for determining and locating a steel pipe mark position based on 3D vision technology, and the method for determining and locating a steel pipe mark position based on 3D vision technology includes:

[0010] Using a robotic arm that integrates a 3D vision device and a laser marking device, perform 3D scanning on the end face of the steel pipe within the field of view of the 3D vision device at the initial position to obtain the first point cloud data;

[0011] Based on the first point cloud data, the point cloud data corresponding to the end face of a single steel pipe is extracted, and the inclination angle of the end face of the steel pipe to be laser marked is calculated. According to the calculated inclination angle of the end face of the steel pipe, the 3D vision device is adjusted to the best shooting posture through the robot arm, and the 3D vision device after the adjusted posture is used to perform 3D scanning again on the end face of the steel pipe to be laser marked to obtain the second point cloud data;

[0012] Based on the second point cloud data, defects on the end surface of the steel pipe are detected;

[0013] Based on the defect detection results and the specifications of the content to be marked, the best marking position is selected on the end face of the steel pipe;

[0014] The position of the laser marking equipment is adjusted by the robotic arm, and the laser marking operation is performed at the optimal marking position.

[0015] Further, based on the first point cloud data, extracting the point cloud data corresponding to the end face of a single steel pipe and calculating the inclination angle of the end face of the steel pipe to be laser marked includes:

[0016] Preprocessing the first point cloud data; wherein the preprocessing includes: firstly reducing the amount of point cloud data by using voxel grid downsampling; and then removing noise points by counting outlier points;

[0017] The preprocessed first point cloud data is divided into multiple clusters by using the DBSCAN point cloud clustering algorithm, wherein each cluster represents the point cloud data corresponding to the end face of a steel pipe;

[0018] Perform plane fitting on each cluster to obtain the equation and normal vector of the plane corresponding to the end face of the corresponding steel pipe;

[0019] Based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser marked, the inclination angle of the end face of the steel pipe to be laser marked is calculated.

[0020] Further, the calculating of the inclination angle of the end face of the steel pipe to be laser marked based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser marked comprises:

[0021] Calculate the angle θ between the normal vector of the plane corresponding to the end face of the steel pipe to be laser marked and the Z-axis direction vector;

[0022] Calculate the tilt angle of the tube end to be laser marked

[0023] Furthermore, the defect detection of the end face of the steel pipe based on the second point cloud data includes:

[0024] Preprocessing the second point cloud data; wherein the preprocessing includes normalization, denoising and downsampling operations;

[0025] For the second point cloud data after preprocessing, the improved point cloud deep learning algorithm PointNet++ is used to identify the size of the steel pipe end face defect and locate the defect.

[0026] Furthermore, the preprocessed second point cloud data is used to identify the size of the steel pipe end surface defect using the improved point cloud deep learning algorithm PointNet++ algorithm and locate the defect, including:

[0027] Use PointNet++ algorithm to obtain the global features of the preprocessed second point cloud data;

[0028] The global features of the preprocessed second point cloud data are compared with the features of the normal point cloud, the differences are calculated, and the features with differences greater than a preset threshold are mapped back to the original point cloud to determine the defect location;

[0029] The defect size is estimated by analyzing the point cloud density and distribution of the defect area.

[0030] Furthermore, the method of selecting the best marking position on the end face of the steel pipe based on the defect detection results and the specifications of the content to be marked includes:

[0031] Calculate the local smoothness of each point in the second point cloud data;

[0032] The size of the sliding window is determined according to the specifications of the content to be marked, and the sliding window is used to search on the second point cloud data. The mean of the local smoothness of all points in each window is calculated as the smoothness value corresponding to the area in the corresponding sliding window, and the area with the highest smoothness value and no defects is selected as the best marking position.

[0033] Further, calculating the local smoothness of each point in the second point cloud data includes:

[0034] Calculate the normal vector of each point in the second point cloud data, including:

[0035] Let the i-th point in the second point cloud data be p i , find p i The neighborhood point set N(p i ), where the number of points in the neighborhood is k;

[0036] Calculate the covariance matrix C of the neighborhood points i :

[0037]

[0038] Among them, pj N(p i ) is the jth point in i Represents the mean of the neighborhood points: T represents the transpose of the matrix;

[0039] C i Perform eigenvalue decomposition and take the eigenvector corresponding to the minimum eigenvalue as p i The normal vector n i ;

[0040] Calculate the local smoothness of each point, including:

[0041] For p i , calculate N(p i ) and n i The difference between

[0042] N(p i ) and n i The mean of the difference between i The local smoothness.

[0043] Furthermore, the laser marking device posture is adjusted by a mechanical arm to perform the laser marking operation at the best marking position, including:

[0044] The position of the laser marking equipment is adjusted by the robotic arm so that the laser marking equipment is aligned with the best marking position and is perpendicular to the end face of the steel pipe to be laser marked, and the laser marking operation is performed at the best marking position.

[0045] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0046] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the above method.

[0047] The technical solution provided by the present invention can ensure the clarity, accuracy and integrity of the end face information during the steel pipe production process, and can accurately and efficiently locate the marking position of the steel pipe end, thereby improving production efficiency and quality stability. At the same time, the present invention also has good adaptability and flexibility, and can adapt to steel pipe ends of different specifications and shapes to meet different production needs. The beneficial effects brought by it include at least:

[0048] 1. The present invention has strong adaptability and can meet the needs of marking tasks for steel pipes of various sizes and types.

[0049] 2. The present invention utilizes high-precision 3D visual equipment to obtain three-dimensional detail information of the end of the steel pipe, and adaptively adjusts the position and posture of the equipment through advanced image processing algorithms and positioning technology.

[0050] 3. The present invention can select the best marking position for marking according to the system marking content and the results of 3D scanning of the steel pipe end face, thus solving the problems of arbitrary position selection and poor marking effect during the marking process.

[0051] 4. The present invention can autonomously identify the shape and size of the end face, and adaptively adjust the position of the marking robot arm according to the identified end face angle, ensuring that the laser is always perpendicular to the end face to be marked, making each marking effect clearer and providing a definite guarantee for future information recognition work. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 3D vision technology-based steel pipe marking position determination and positioning method provided in an embodiment of the present invention;

[0054] Figure 2 It is a technical flow chart of a method for determining and locating a steel pipe mark position based on 3D vision technology provided by an embodiment of the present invention;

[0055] Figure 3 It is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0057] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0058] First embodiment

[0059] This embodiment provides a method for determining and locating the position of a steel pipe mark based on 3D vision technology. The method can be implemented by an electronic device, which can be a terminal or a server. The implementation process and technical flow of the method are as follows: Figure 1 and Figure 2 As shown, it includes four parts: ① The robot performs 3D scanning on the end face of the steel pipe. ② The system identifies defects after image processing based on the scanning results. ③ The robot selects the marking position based on the defect position and the system marking content. ④ The robot adjusts its posture and performs the marking task.

[0060] Specifically, the execution process of the method includes the following steps:

[0061] S1, using a robotic arm integrating a 3D vision device and a laser marking device, 3D scan the end face of the steel pipe within the field of view of the 3D vision device at the initial position to obtain the first point cloud data;

[0062] It should be noted that the above S1 utilizes a robotic arm integrating 3D vision and laser marking to take pictures of objects within a certain range at the initial position of the camera, so as to perform subsequent end face recognition and approximate positioning.

[0063] S2, based on the first point cloud data, extract the point cloud data corresponding to the end face of a single steel pipe, and calculate the inclination angle of the end face of the steel pipe to be laser marked, according to the calculated inclination angle of the end face of the steel pipe, adjust the 3D vision device to the best shooting posture through the robot arm, and use the 3D vision device after the adjusted posture to perform 3D scanning again on the end face of the steel pipe to be laser marked, to obtain the second point cloud data;

[0064] Among them, it should be noted that the above S2 uses image segmentation and image recognition algorithms to separate the approximate position of the object to be identified from the image, uses point cloud clustering to extract the end face information of the pipe head, calculates the inclination angle of the end face of the steel pipe, and uses the calculated angle and position information to plan the overall path of the robotic arm. The robotic arm adjusts the camera posture according to the planned path, selects the best shooting position and angle (vertical end face), and uses high-precision 3D vision equipment to scan the end of the steel pipe to obtain its detailed three-dimensional image data.

[0065] Specifically, in this embodiment, the implementation process of the above S2 is as follows:

[0066] S21, read point cloud data p i =(x i ,y i ,z i), preprocessing the first point cloud data; wherein the preprocessing operation includes: firstly using voxel grid downsampling to reduce the amount of point cloud data so as to speed up the processing speed; and then removing noise points by counting outliers to improve data quality;

[0067] S22, dividing the preprocessed first point cloud data into a plurality of clusters by using a DBSCAN point cloud clustering algorithm, each cluster representing point cloud data corresponding to an end face of a steel pipe, that is, each cluster represents a plane or an object;

[0068] S23, plane fitting is performed on each cluster (i.e., a group of point clouds) to obtain the equation and normal vector of the plane corresponding to the end face of the corresponding steel pipe, specifically:

[0069] S231: Calculate the centroid:

[0070]

[0071] Where, c: center of mass;

[0072] N: number of points;

[0073] p i =(x i ,y i ,z i ): the i-th point in the point cloud;

[0074] S232: Building a decentralized matrix:

[0075] P′=Pc

[0076] Where, P: original point cloud matrix;

[0077] P′: decentralized point cloud matrix;

[0078] S233: SVD decomposition (singular value decomposition):

[0079] P′=U∑V T

[0080] Where, U, ∑, V: matrices after SVD decomposition;

[0081] S234: Extract plane normal vector:

[0082] The plane normal vector is the third column of V (the eigenvector corresponding to the smallest singular value);

[0083] S235: Obtain the plane equation:

[0084] ax+by+cz+d=0

[0085] Among them, the normal vector n = (a, b, c) is the last column of eigenvectors obtained by SVD decomposition;

[0086] S24, based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser marked, calculate the inclination angle of the end face of the steel pipe to be laser marked, specifically:

[0087] S241, calculate the angle θ between the normal vector and the Z-axis direction vector:

[0088]

[0089] Where n = (a, b, c): normal vector;

[0090] z=(0,0,1): vector in the Z-axis direction;

[0091] n·z=c: dot product;

[0092] The modulus of the normal vector

[0093]

[0094] Where, θ: angle;

[0095] angle_degrees: angle (degrees);

[0096] S25, performing overall path planning of the robot arm based on the calculated angle and position information;

[0097] S26, adjust the camera posture according to the planned path through the robotic arm, select the best shooting position and angle, and use the 3D vision equipment to perform a second scan on the end of the steel pipe to obtain its detailed three-dimensional image data.

[0098] S3, detecting defects on the end surface of the steel pipe based on the second point cloud data;

[0099] Specifically, in this embodiment, the implementation process of the above S3 is as follows:

[0100] S31, preprocessing the second point cloud data; wherein the preprocessing includes normalization, denoising, downsampling and other operations to improve the image data quality; the details are as follows:

[0101] Normalization:

[0102]

[0103] Where, c: point cloud centroid,

[0104] N: number of points;

[0105] p i =(x i ,y i ,z i ): the i-th point in the point cloud;

[0106] d max : The maximum distance from all points to the centroid c, that is:

[0107] Through the above preprocessing process, the distances of all points are scaled to the maximum distance d max The entire point cloud is scaled to the unit sphere, keeping the overall geometric shape and size of the point cloud unchanged, retaining the overall geometric information, which is suitable for local and global feature extraction of PointNet++.

[0108] S32, for the second point cloud data after preprocessing, the improved point cloud deep learning algorithm PointNet++ algorithm is used to identify the size of the end surface defect of the steel pipe and locate the defect, as follows:

[0109] S321: Stratified Sampling:

[0110] Firstly, the FPS (Farthest Point Sampling) algorithm is used for hierarchical sampling to select representative points from the original point cloud, thereby reducing the computational complexity, enhancing the real-time performance of the system and retaining the overall structure of the point cloud.

[0111] Randomly select a point p from the point cloud P i As the first sampling point, add it to the sampling set P'. For each point to be sampled, calculate its distance to the nearest point in the sampled point set P'. Select the point with the largest distance as the next sampling point, add it to P', and repeat the above steps until the required number of sampling points are selected.

[0112] P′={p i ∣p i ∈P and p i is selected by FPS}

[0113] Where, P: original point cloud;

[0114] P′: is the point set after sampling;

[0115] S322: Grouping:

[0116] Select representative points from the point cloud step by step. For each sampling point, select a group of points in its neighborhood and use the spherical area query algorithm to perform feature aggregation:

[0117]

[0118] in, Indicates p i The neighborhood set of ;

[0119] r is the area query radius;

[0120] S323: Local Feature Learning:

[0121] In each neighborhood, the PointNet++ architecture is used to learn the local features, and the neural network is used to extract features from each local point cloud subset to generate local features.

[0122] For each local point cloud subset Calculate each neighborhood point relative to the center point p i The offset p j -p i .

[0123] Use multi-layer perceptron (MLP) to extract features from the offset and obtain the local feature h(p j -p i ,f j ), where f j It's point p j original features.

[0124] Perform maximum pooling on all local features to obtain the local point p i The final feature f i :

[0125]

[0126] Where h is a multi-layer perceptron (MLP) and γ is the maximum pooling function;

[0127] S324: Hierarchical Aggregation:

[0128] Aggregate local features at multiple levels, aggregate features layer by layer through multiple steps, and finally get the global feature representation:

[0129] The local features of the first layer are used as input to perform sampling, grouping and feature extraction on the next layer to obtain higher-level local features.

[0130] Repeat the above steps to aggregate features layer by layer.

[0131] In the last layer, all local features are globally pooled to obtain the global feature F of the entire point cloud:

[0132] F=γ({f i ∣p i ∈P′})

[0133] S325: Compare the features of the second point cloud data with the features of the normal point cloud and calculate the difference:

[0134] D=∥F normal -F test ∥

[0135] Where, D: the difference between global features;

[0136] F normal : Global features of normal point cloud;

[0137] F test : Global features of the second point cloud data;

[0138] S326: Compare the global features of the preprocessed second point cloud data with the features of the normal point cloud, calculate the difference, map the features whose difference is greater than a preset threshold τ back to the original point cloud, and determine the defect position;

[0139] S327: Estimate the defect size by analyzing the point cloud density and distribution of the defect area.

[0140] S4, based on the defect detection results and the specifications of the content to be marked, select the best marking position on the end face of the steel pipe;

[0141] Among them, it should be noted that the above S4 is based on the high-definition three-dimensional image obtained by the system to accurately capture the three-dimensional information of the end of the steel pipe, including the contour, convexity and concaveness of the end, etc., set different traversal areas according to different specifications of the marking content, use the point cloud angle to identify the smoother area, and select the best marking position.

[0142] Specifically, in this embodiment, the implementation process of the above S4 is as follows:

[0143] S41, calculating the local smoothness of each point in the second point cloud data;

[0144] It should be noted that, in order to obtain the smoothness of each point in its local area, this embodiment uses the normal vector consistency to determine whether the normal vectors of each point and its neighboring points in the point cloud are consistent, thereby reflecting the smoothness of the area where the point is located. Therefore, by calculating the normal vector consistency of each point and its neighboring points, the smoothness of the small area where the point is located is determined. The specific process is as follows:

[0145] S411: Normal vector consistency calculation: Calculate the normal vector consistency of each point as the smoothness value.

[0146] Among them, normal vector consistency measures the consistency between normal vectors in a certain area of ​​the point cloud. If the normal vectors of a region are roughly the same, then the region is smooth.

[0147] First, calculate the normal vector for each point:

[0148] Let the i-th point in the point cloud be p i .

[0149] Find point p i The neighborhood point set N(p i ), the neighborhood radius is r or the number of neighborhood points is k.

[0150] Calculate the covariance matrix C of the neighborhood points i :

[0151]

[0152] Among them, μ i is the mean of the neighborhood points:

[0153]

[0154] The covariance matrix C i Perform eigenvalue decomposition and take the eigenvector corresponding to the minimum eigenvalue as point p i The normal vector n i ;

[0155] S412: Normal vector consistency calculation:

[0156] For each point p i , calculate the difference between the normal vector of its neighboring points and the normal vector of this point.

[0157] Smoothness is defined as the average of these differences:

[0158]

[0159] Among them, n i : The normal vector of the i-th point;

[0160] Point p i The set of neighboring points of ;

[0161] k: the number of points in the neighborhood;

[0162] S42, determining the size of the sliding window according to the specifications of the content to be marked, using the sliding window to search on the second point cloud data, calculating the mean of the local smoothness of all points in each window as the smoothness value corresponding to the corresponding area, and selecting the area with the highest smoothness value and no defects as the best marking position;

[0163] It should be noted that in order to find the best area that matches the system content specifications in the entire point cloud data, the size of the area to be searched in the image needs to be determined according to the marking content specifications issued by the system. Then use a sliding window to search on the point cloud data, and calculate the smoothness in each window. By summarizing the local smoothness of all points in the window, calculate the overall smoothness of the window, and evaluate the entire window, the area with the highest smoothness is found as the best area that matches the system content specifications. The specific process is as follows:

[0164] S421: define the sliding window size: the window size is W×H;

[0165] S422: Sliding window traversal:

[0166] Slide the window with step sizes Δx and Δy within the range of the point cloud data;

[0167] Assume the coordinates of the upper left corner of the window are (x,y), then the window range is (x,y+W)×(y,y+H);

[0168] S423: Smoothness calculation:

[0169] For each window, calculate the smoothed mean of all points within the window:

[0170]

[0171] Where: N: the number of points in the window;

[0172] Window(x,y): represents the set of points within the window range;

[0173] S424: Select the best matching position:

[0174] According to the smoothness value in the sliding window, the area with the highest smoothness is selected as the best matching position:

[0175]

[0176] S5, adjusting the position of the laser marking equipment through the robotic arm, and performing the laser marking operation at the optimal marking position;

[0177] Specifically, in this embodiment, the above S5 is specifically:

[0178] The position of the laser marking equipment is adjusted by the robotic arm so that the laser marking equipment is aligned with the best marking position and is perpendicular to the end face of the steel pipe to be laser marked, and the laser marking operation is performed at the best marking position.

[0179] In summary, this embodiment provides a method for determining and locating the steel pipe marking position based on 3D vision technology. The method uses high-precision 3D vision equipment to obtain three-dimensional detail information of the end of the steel pipe, and uses advanced image processing algorithms and positioning technology to adaptively adjust the position and posture of the equipment so that the laser equipment is always perpendicular to the end face, identify end face defects, and select the best laser marking position on the end face that meets the marking conditions according to the length of the marking content issued by the system. This solves the problems of arbitrary position selection and poor marking effect during laser marking, and can meet the needs of marking tasks for steel pipes of multiple sizes and types. It improves the marking efficiency of the entire industry and ensures the accuracy and recognition rate of information.

[0180] Second embodiment

[0181] This embodiment provides an electronic device, such as Figure 3 As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0182] Next, combine Figure 3 The following is a detailed introduction to the various components of the electronic device:

[0183] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0184] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 3 The CPU0 and CPU1 shown in the figure are, of course, only exemplary.

[0185] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0186] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0187] The transceiver may include a receiver and a transmitter ( Figure 3 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0188] In addition, it should be noted that Figure 3 The structure of the electronic device shown in the figure does not constitute a limitation on the device, and the actual device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment above can refer to the technical effects described in the first embodiment above, so they are not repeated here.

[0189] Third embodiment

[0190] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.

[0191] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiment of the present invention can be in the form of a full or partial hardware embodiment, a full or partial software embodiment or an embodiment combining software and hardware. Moreover, when implemented using software, the embodiment of the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state hard disk.

[0192] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0193] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0194] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. In addition, the term "and / or" is only an association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc or abc, where a, b, c can be single or plural.

[0195] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0196] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0197] In several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0198] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0199] Finally, it should be noted that the above is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for ordinary technicians in this technical field, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A method for determining and locating the position of a steel pipe mark based on 3D vision technology, characterized in that: include: Using a robotic arm that integrates a 3D vision device and a laser marking device, perform 3D scanning on the end face of the steel pipe within the field of view of the 3D vision device at the initial position to obtain the first point cloud data; Based on the first point cloud data, the point cloud data corresponding to the end face of a single steel pipe is extracted, and the inclination angle of the end face of the steel pipe to be laser marked is calculated. According to the calculated inclination angle of the end face of the steel pipe, the 3D vision device is adjusted to the best shooting posture through the robot arm, and the 3D vision device after the adjusted posture is used to perform 3D scanning again on the end face of the steel pipe to be laser marked to obtain the second point cloud data; Based on the second point cloud data, defects on the end surface of the steel pipe are detected; Based on the defect detection results and the specifications of the content to be marked, the best marking position is selected on the end face of the steel pipe; The position of the laser marking equipment is adjusted by the robotic arm, and the laser marking operation is performed at the optimal marking position.

2. The method for determining and locating the steel pipe marking position based on 3D vision technology according to claim 1, characterized in that: The method of extracting point cloud data corresponding to a single steel pipe end face based on the first point cloud data and calculating the inclination angle of the steel pipe end face to be laser marked includes: Preprocessing the first point cloud data; wherein the preprocessing includes: firstly reducing the amount of point cloud data by using voxel grid downsampling; and then removing noise points by counting outlier points; The preprocessed first point cloud data is divided into multiple clusters by using the DBSCAN point cloud clustering algorithm, wherein each cluster represents the point cloud data corresponding to the end face of a steel pipe; Perform plane fitting on each cluster to obtain the equation and normal vector of the plane corresponding to the end face of the corresponding steel pipe; Based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser marked, the inclination angle of the end face of the steel pipe to be laser marked is calculated.

3. The method for determining and locating the steel pipe marking position based on 3D vision technology according to claim 2, characterized in that: The step of calculating the inclination angle of the end face of the steel pipe to be laser marked based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser marked comprises: Calculate the angle θ between the normal vector of the plane corresponding to the end face of the steel pipe to be laser marked and the Z-axis direction vector; Calculate the tilt angle of the tube end to be laser marked 4. The method for determining and locating the steel pipe marking position based on 3D vision technology according to claim 1, characterized in that: The method of detecting defects on the end surface of the steel pipe based on the second point cloud data includes: Preprocessing the second point cloud data; wherein the preprocessing includes normalization, denoising and downsampling operations; For the second point cloud data after preprocessing, the improved point cloud deep learning algorithm PointNet++ is used to identify the size of the steel pipe end face defect and locate the defect.

5. The method for determining and locating the steel pipe marking position based on 3D vision technology according to claim 4, characterized in that: The method of using the improved point cloud deep learning algorithm PointNet++ to identify the size of the steel pipe end surface defect and locate the defect for the preprocessed second point cloud data includes: Use PointNet++ algorithm to obtain the global features of the preprocessed second point cloud data; The global features of the preprocessed second point cloud data are compared with the features of the normal point cloud, the differences are calculated, and the features with differences greater than a preset threshold are mapped back to the original point cloud to determine the defect location; The defect size is estimated by analyzing the point cloud density and distribution of the defect area.

6. The method for determining and locating the steel pipe marking position based on 3D vision technology according to claim 1, characterized in that: The method of selecting the best marking position on the end face of the steel pipe based on the defect detection results and the specifications of the content to be marked includes: Calculate the local smoothness of each point in the second point cloud data; The size of the sliding window is determined according to the specifications of the content to be marked, and the sliding window is used to search on the second point cloud data. The mean of the local smoothness of all points in each window is calculated as the smoothness value corresponding to the area in the corresponding sliding window, and the area with the highest smoothness value and no defects is selected as the best marking position.

7. The method for determining and locating the steel pipe marking position based on 3D vision technology according to claim 6, characterized in that: The calculating the local smoothness of each point in the second point cloud data includes: Calculate the normal vector of each point in the second point cloud data, including: Let the i-th point in the second point cloud data be p i , find p i The neighborhood point set N(p i ), where the number of points in the neighborhood is k; Calculate the covariance matrix C of the neighborhood points i : Among them, p j N(p i ) is the jth point in i Represents the mean of the neighborhood points: T represents the transpose of the matrix; C i Perform eigenvalue decomposition and take the eigenvector corresponding to the minimum eigenvalue as p i The normal vector n i ; Calculate the local smoothness of each point, including: For p i , calculate N(p i ) and n i The difference between N(p i ) and n i The mean of the difference between i The local smoothness.

8. The method for determining and locating the steel pipe marking position based on 3D vision technology according to claim 1, characterized in that: The method of adjusting the posture of the laser marking device by a mechanical arm and performing the laser marking operation at the optimal marking position includes: The position of the laser marking equipment is adjusted by the robotic arm so that the laser marking equipment is aligned with the best marking position and is perpendicular to the end face of the steel pipe to be laser marked, and the laser marking operation is performed at the best marking position.

Images