A steel pipe marking position judgment and positioning method based on 3D vision technology

Abstract

This invention discloses a method for determining and locating the marking position of steel pipes based on 3D vision technology, belonging to the fields of industrial automation and machine vision technology. The method includes: performing a 3D scan of the steel pipe end face within a certain range from the initial position of the camera to acquire first point cloud data; calculating the tilt angle of the steel pipe end face based on the first point cloud data, adjusting the camera to the optimal shooting posture, and performing a second 3D scan of the steel pipe end face to acquire second point cloud data; performing defect detection on the steel pipe end face based on the second point cloud data; selecting the optimal marking position on the steel pipe end face based on the defect detection results and the specifications of the content to be marked; adjusting the posture of the laser marking equipment and performing laser marking operation at the optimal marking position. This invention can meet the needs of marking tasks for steel pipes of various sizes and types, improve the overall marking efficiency of the industry, and ensure the accuracy and recognition rate of information.

Description

Technical Field

[0001] This invention relates to the fields of industrial automation and machine vision technology, and in particular to a method for determining and locating the position of steel pipe markings based on 3D vision technology. Background Technology

[0002] In the steel pipe production process, the markings on the steel pipes often contain a lot of important information (including billet number, furnace number, specifications, etc.), which are the main basis for material tracking and product traceability.

[0003] Marking steel pipes is a crucial step in ensuring product quality, enabling production traceability, and meeting quality inspection requirements. The selection of the marking location and the accuracy of the marking information are of paramount importance to subsequent production, quality inspection, and traceability processes.

[0004] However, traditional marking position positioning methods mainly rely on manual operation or simple 2D image processing technology. These methods are often easily affected by factors such as environment, product specifications and dimensions, and marking position quality, resulting in unstable positioning accuracy and low efficiency.

[0005] In traditional laser marking processes, the laser is not guaranteed to be strictly perpendicular to the end face. Due to the arbitrariness in the selection of marking angles, the marking information often has varying depths and clarity. The non-standard laser marking angles also cause problems such as distortion and overlap of the marking information, which seriously affects subsequent information recognition and tracking tasks.

[0006] Traditional laser marking processes do not select a marking area, relying instead on direct marking. This results in unclear markings of defective areas and a high risk of identification errors. Furthermore, a mismatch between the marking content and the end face size can lead to errors where the marking area is insufficient to fully display the content. Summary of the Invention

[0007] This invention provides a method for determining and locating the position of steel pipe markings based on 3D vision technology, in order to solve the technical problems of arbitrary position selection and poor marking effect in the laser marking process of existing technologies.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0009] On one hand, the present invention provides a method for determining and locating the position of steel pipe markings based on 3D vision technology, the method comprising:

[0010] Using a robotic arm that integrates 3D vision equipment and laser marking equipment, the end face of the steel pipe within the field of view of the 3D vision equipment is 3D scanned at the initial position to obtain the first point cloud data;

[0011] Based on the first point cloud data, the point cloud data corresponding to a single steel pipe end face is extracted, and the tilt angle of the steel pipe end face to be laser marked is calculated. According to the calculated tilt angle of the steel pipe end face, the 3D vision device is adjusted to the best shooting position by the robotic arm, and the 3D vision device after the position adjustment is used to perform 3D scanning on the steel pipe end face to be laser marked again to obtain the second point cloud data.

[0012] Based on the second point cloud data, defects on the end face of the steel pipe are detected;

[0013] Based on the defect detection results and the specifications of the content to be marked, the optimal marking position is selected on the end face of the steel pipe.

[0014] The laser marking equipment is positioned and oriented by a robotic arm to perform laser marking at the optimal marking location.

[0015] Further, the step of extracting the point cloud data corresponding to a single steel pipe end face based on the first point cloud data and calculating the tilt angle of the steel pipe end face to be laser marked includes:

[0016] The first point cloud data is preprocessed; wherein, the preprocessing includes: firstly, using voxel grid downsampling to reduce the amount of point cloud data; and then removing noise points by statistical outlier removal.

[0017] The preprocessed first point cloud data is divided into multiple clusters using the DBSCAN point cloud clustering algorithm, where each cluster represents the point cloud data corresponding to a steel pipe end face.

[0018] For each cluster, perform plane fitting to obtain the equation and normal vector of the plane corresponding to the end face of the steel pipe;

[0019] Based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser-marked, the tilt angle of the end face of the steel pipe to be laser-marked is calculated.

[0020] Further, the calculation of the tilt angle of the steel pipe end face to be laser-marked, based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser-marked, includes:

[0021] Calculate the angle θ between the normal vector of the plane corresponding to the end face of the steel pipe to be laser marked and the Z-axis direction vector;

[0022] Calculate the tilt angle of the steel pipe end face to be laser marked.

[0023] Furthermore, the detection of defects on the steel pipe end face based on the second point cloud data includes:

[0024] The second point cloud data is preprocessed; the preprocessing includes normalization, noise reduction and downsampling operations.

[0025] For the preprocessed second point cloud data, the improved PointNet++ deep learning algorithm is used to identify the size of the defects on the steel pipe end face and to locate the defects.

[0026] Furthermore, the step of identifying the size of the defect on the steel pipe end face and locating the defect using the improved PointNet++ deep learning algorithm for the preprocessed second point cloud data includes:

[0027] The PointNet++ algorithm is used to obtain the global features of the preprocessed second point cloud data.

[0028] The global features of the preprocessed second point cloud data are compared with the features of the normal point cloud. The difference is calculated, and features with a difference greater than a preset threshold are mapped back to the original point cloud to determine the defect location.

[0029] The size of the defect is estimated by analyzing the point cloud density and distribution in the defect area.

[0030] Furthermore, the step of selecting the optimal marking position on the end face of the steel pipe based on the defect detection results and the specifications of the content to be marked includes:

[0031] Calculate the local smoothness of each point in the second point cloud data;

[0032] The size of the sliding window is determined based on the specifications of the content to be labeled. The sliding window is used to search the second point cloud data. The mean of the local smoothness of all points in each window is calculated as the smoothness value of the corresponding area in the sliding window. The area with the highest smoothness value and no defects is selected as the best labeling position.

[0033] Further, the calculation of the local smoothness of each point in the second point cloud data includes:

[0034] Calculate the normal vector of each point in the second point cloud data, including:

[0035] Let p be the i-th point in the second point cloud data. i Find p i The set of neighborhood points N(p) i ); where the number of points in the neighborhood is k;

[0036] Calculate the covariance matrix C of the neighborhood points i :

[0037]

[0038] Where, pj N(p) i The j-th point within ); μ i The mean of the neighborhood points is represented by T, which represents the transpose of the matrix.

[0039] For C i Perform eigenvalue decomposition and take the eigenvector corresponding to the smallest eigenvalue as p. i normal vector n i ;

[0040] Calculate the local smoothness at each point, including:

[0041] For p i Calculate N(p) i The normal vectors of each point within n) and n i The differences between them;

[0042] With N(p i The normal vectors of each point within n) and n i The mean of the differences between them is taken as p. i Local smoothness.

[0043] Furthermore, the step of adjusting the position of the laser marking equipment using a robotic arm to perform laser marking at the optimal marking position includes:

[0044] The position of the laser marking equipment is adjusted by the robotic arm so that the laser marking equipment is aligned with the optimal marking position and perpendicular to the end face of the steel pipe to be laser marked, and the laser marking operation is performed at the optimal marking position.

[0045] In another aspect, the present invention also provides an electronic device comprising a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above-described method.

[0046] In another aspect, the present invention also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the above method.

[0047] The technical solution provided by this invention ensures the clarity, accuracy, and completeness of end-face information during steel pipe production, enabling precise and efficient positioning of the marking location on the steel pipe end, thereby improving production efficiency and quality stability. Furthermore, this invention possesses good adaptability and flexibility, accommodating steel pipe ends of different specifications and shapes to meet diverse production needs. Its beneficial effects include at least the following:

[0048] 1. This invention has strong adaptability and can meet the needs of marking various sizes and types of steel pipes.

[0049] 2. This invention utilizes high-precision 3D vision equipment to acquire three-dimensional detail information of the end of the steel pipe, and adaptively adjusts the position and posture of the equipment through advanced image processing algorithms and positioning technology.

[0050] 3. This invention can select the best marking position for marking based on the marking content of the system and the results of 3D scanning of the steel pipe end face, thus solving the problems of arbitrary position selection and poor marking effect during the marking process.

[0051] 4. This invention can autonomously identify the shape and size of the end face and adaptively adjust the posture of the marking robot arm according to the identified end face angle, ensuring that the laser is always perpendicular to the end face to be marked, making the marking effect clearer each time and providing a definite guarantee for future information recognition work. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram illustrating the implementation of the method for determining and locating the position of steel pipe markings based on 3D vision technology provided in this embodiment of the invention.

[0054] Figure 2 This is a technical flowchart of the method for determining and locating the position of steel pipe markings based on 3D vision technology provided in this embodiment of the invention;

[0055] Figure 3 This is a system block diagram of the electronic device provided in the embodiments of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0057] First, it should be noted that in the embodiments of the present invention, the words "exemplarily," "for example," etc., are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the term "exemplarily" is intended to present the concept in a specific manner. Furthermore, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either one or the other.

[0058] First Embodiment

[0059] This embodiment provides a method for determining and locating the position of steel pipe markings based on 3D vision technology. This method can be implemented using an electronic device, which can be a terminal or a server. The implementation process and technical flow of this method are as follows: Figure 1 and Figure 2 As shown, the process includes four parts: ① The robot performs a 3D scan of the steel pipe end face. ② Based on the scan results, the system processes the image and identifies defects. ③ The robot selects the marking location based on the defect location and the system's marking information. ④ The robot adjusts its posture and performs the marking task.

[0060] Specifically, the execution flow of this method includes the following steps:

[0061] S1, using a robotic arm that integrates 3D vision equipment and laser marking equipment, performs 3D scanning on the end face of the steel pipe within the field of view of the 3D vision equipment at the initial position to obtain the first point cloud data;

[0062] It should be noted that S1 above uses a robotic arm that integrates 3D vision and laser marking to take pictures of objects within a certain range at the initial position of the camera, thereby enabling subsequent end-face recognition and approximate positioning.

[0063] S2, based on the first point cloud data, extract the point cloud data corresponding to a single steel pipe end face, and calculate the tilt angle of the steel pipe end face to be laser marked. According to the calculated tilt angle of the steel pipe end face, the 3D vision device is adjusted to the best shooting position by the robotic arm, and the 3D vision device after the position adjustment is used to perform 3D scanning on the steel pipe end face to be laser marked again to obtain the second point cloud data.

[0064] It should be noted that S2 above uses image segmentation and image recognition algorithms to extract the approximate position of the object to be identified from the image, uses point cloud clustering to extract the end face information of the pipe, calculates the tilt angle of the steel pipe end face, and uses the calculated angle and position information to plan the overall path of the robotic arm. The robotic arm adjusts the camera posture according to the planned path, selects the best shooting position and angle (vertical end face), and uses high-precision 3D vision equipment to scan the end of the steel pipe to obtain its detailed three-dimensional image data.

[0065] Specifically, in this embodiment, the implementation process of S2 is as follows:

[0066] S21, Read point cloud data p i =(x i ,y i ,z iThe first point cloud data is preprocessed. The preprocessing operations include: firstly, using voxel grid downsampling to reduce the amount of point cloud data in order to speed up the processing speed; and then removing noise points by statistical outlier removal to improve data quality.

[0067] S22, the preprocessed first point cloud data is divided into multiple clusters by the DBSCAN point cloud clustering algorithm. Each cluster represents the point cloud data corresponding to a steel pipe end face, that is, each cluster represents a plane or object.

[0068] S23, perform plane fitting on each cluster (i.e., a group of point clouds) to obtain the equation and normal vector of the plane corresponding to the end face of the steel pipe, specifically:

[0069] S231: Calculate the centroid:

[0070]

[0071] Where c: centroid;

[0072] N: The number of points;

[0073] p i =(x i ,y i ,z i ): The i-th point in the point cloud;

[0074] S232: Constructing a decentralized matrix:

[0075] P′=Pc

[0076] Where P: original point cloud matrix;

[0077] P′: Decentralized point cloud matrix;

[0078] S233: SVD Decomposition (Singular Value Decomposition):

[0079] P′=U∑V T

[0080] Where U, ∑, V: matrices after SVD decomposition;

[0081] S234: Extract the plane normal vector:

[0082] The plane normal vector is the third column of V (the eigenvector corresponding to the minimum singular value);

[0083] S235: The equation of the plane is derived:

[0084] ax + by + cz + d = 0

[0085] Wherein, the normal vector n = (a, b, c) is the last column of eigenvectors obtained by SVD decomposition;

[0086] S24, based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser-marked, calculate the tilt angle of the end face of the steel pipe to be laser-marked, specifically:

[0087] S241, Calculate the angle θ between the normal vector and the Z-axis direction vector:

[0088]

[0089] Where n = (a, b, c): normal vector;

[0090] z = (0,0,1): Z-axis direction vector;

[0091] n·z=c: dot product;

[0092] Magnitude of the normal vector

[0093]

[0094] Where θ: included angle;

[0095] angle_degrees: Angles (in degrees);

[0096] S25 uses the calculated angle and position information to perform overall path planning for the robotic arm;

[0097] S26 uses a robotic arm to adjust the camera posture according to a planned path, selects the best shooting position and angle, and uses 3D vision equipment to perform a second scan on the end of the steel pipe to obtain its detailed three-dimensional image data.

[0098] S3, based on the second point cloud data, detects defects on the end face of the steel pipe;

[0099] Specifically, in this embodiment, the implementation process of S3 is as follows:

[0100] S31, preprocess the second point cloud data; wherein, the preprocessing includes operations such as normalization, noise reduction, and downsampling to improve the image data quality; specifically as follows:

[0101] Normalization:

[0102]

[0103] Where c: centroid of the point cloud,

[0104] N: The number of points;

[0105] p i =(x i ,y i ,z i ): The i-th point in the point cloud;

[0106] d max The maximum distance from all points to the centroid c, i.e.:

[0107] Through the above preprocessing steps, the distances of all points are scaled to the maximum distance d. max Within a unit sphere, the entire point cloud is scaled down to the size of the unit sphere, maintaining the overall geometry and size of the point cloud and preserving the overall geometric information, making it suitable for local and global feature extraction in PointNet++.

[0108] S32, for the preprocessed second point cloud data, the improved PointNet++ point cloud deep learning algorithm is used to identify the size of the defects on the steel pipe end face and to locate the defects, as detailed below:

[0109] S321: Layered Sampling

[0110] First, the FPS (Farthest Point Sampling) algorithm is used for hierarchical sampling to select representative points from the original point cloud, reducing computational complexity, enhancing system real-time performance, and preserving the overall structure of the point cloud.

[0111] Randomly select a point p from the point cloud P. i As the first sampling point, add it to the sampling set P′. For each sampling point, calculate its distance to the nearest point in the already sampled point set P′. Select the point with the largest distance as the next sampling point and add it to P′. Repeat the above steps until the required number of sampling points are selected.

[0112] P′={p i |p i ∈P and p i is selected by FPS}

[0113] Wherein, P: original point cloud;

[0114] P′: is the sampled point set;

[0115] S322: Grouping

[0116] Representative points are selected progressively from the point cloud. For each sampled point, a set of points within its neighborhood are selected for feature aggregation using a spherical region query algorithm.

[0117]

[0118] in, p i The neighborhood set;

[0119] r is the range of the region query;

[0120] S323: Local Feature Learning

[0121] Within each neighborhood, local features are learned using the PointNet++ architecture, and features are extracted from each local point cloud subset using a neural network to generate local features.

[0122] For each local point cloud subset Calculate the relationship between each neighboring point and the center point p. i offset p j -p i .

[0123] The offset is used to extract features, resulting in local features h(p). j -p i ,f j ), where f j It is point p j The original characteristics.

[0124] Max pooling is performed on all local features to obtain the local point p. i The final feature f i :

[0125]

[0126] Where h is a multilayer perceptron (MLP) and γ is the max pooling function;

[0127] S324: Hierarchical Aggregation

[0128] By aggregating local features at multiple levels and repeating the above steps layer by layer, a global feature representation is finally obtained.

[0129] Using the local features of the first layer as input, the next layer performs sampling, grouping, and feature extraction to obtain higher-level local features.

[0130] Repeat the above steps to aggregate features layer by layer.

[0131] In the final layer, all local features are subjected to global max pooling to obtain the global features F of the entire point cloud:

[0132] F=γ({f i |p i ∈P′})

[0133] S325: Compare the features of the second point cloud data with the features of the normal point cloud, and calculate the differences:

[0134] D = ∥ F normal -F test ∥

[0135] Where D represents the difference between global features;

[0136] F normal : Global features of a normal point cloud;

[0137] F test The second point is the global characteristics of cloud data;

[0138] S326: Compare the global features of the preprocessed second point cloud data with the features of the normal point cloud, calculate the difference, and map the features with a difference greater than the preset threshold τ back to the original point cloud to determine the defect location.

[0139] S327: Estimate the size of the defect by analyzing the point cloud density and distribution in the defect area.

[0140] S4. Based on the defect detection results and the specifications of the content to be marked, select the best marking position on the end face of the steel pipe.

[0141] It should be noted that S4 above is based on the high-definition three-dimensional image obtained by the system to accurately capture the three-dimensional information of the end of the steel pipe, including the contour of the end, the concavity and convexity, etc. Different traversal areas are set according to the different specifications of the marking content, and the smoother areas are identified by point cloud angles to select the best marking position.

[0142] Specifically, in this embodiment, the implementation process of S4 is as follows:

[0143] S41, calculate the local smoothness of each point in the second point cloud data;

[0144] It should be noted that, in order to obtain the smoothness of each point within its local region, this embodiment utilizes normal vector consistency to determine whether the normal vectors of each point and its neighboring points in the point cloud are consistent, thereby reflecting the smoothness of the region where that point is located. Therefore, by calculating the normal vector consistency of each point and its neighboring points, the smoothness of the small region where that point is located is determined. The specific process is as follows:

[0145] S411: Normal Vector Consistency Calculation: Calculate the normal vector consistency of each point as a smoothness value.

[0146] Normal vector consistency measures the degree of consistency among the normal vectors of a region in a point cloud. If the normal vectors of a region are roughly the same, then that region is considered smooth.

[0147] First, calculate the normal vector for each point:

[0148] Let p be the i-th point in the point cloud. i .

[0149] Find point p i The set of neighborhood points N(p) i ), where the neighborhood radius is r or the number of neighborhood points is k.

[0150] Calculate the covariance matrix C of the neighborhood points i :

[0151]

[0152] Where, μ i It is the mean of the neighborhood points:

[0153]

[0154] For the covariance matrix C i Perform eigenvalue decomposition and take the eigenvector corresponding to the smallest eigenvalue as the point p. i normal vector n i ;

[0155] S412: Normal Vector Consistency Calculation:

[0156] For each point p i Calculate the difference between the normal vector of its neighboring points and the normal vector of the point itself.

[0157] Smoothness is defined as the average of these differences:

[0158]

[0159] Where, n i : The normal vector of the i-th point;

[0160] Point p i The set of neighborhood points;

[0161] k: The number of points in the neighborhood;

[0162] S42, determine the size of the sliding window according to the specifications of the content to be identified, use the sliding window to search on the second point cloud data, calculate the mean of the local smoothness of all points in each window, use it as the smoothness value corresponding to the corresponding area, and select the area with the highest smoothness value and no defects as the best identification position.

[0163] It's important to note that to find the optimal region matching the system's content specifications within the entire point cloud data, the size of the region to be searched in the image needs to be determined based on the labeling specifications issued by the system. Then, a sliding window is used to search the point cloud data, and the smoothness within each window is calculated. By summing the local smoothness of all points within a window, the overall smoothness of the window is calculated, and the entire window is evaluated to find the region with the highest smoothness as the optimal region matching the system's content specifications. The specific process is as follows:

[0164] S421: Define the sliding window size: the window size is W×H;

[0165] S422: Sliding window traversal:

[0166] Within the range of point cloud data, a sliding window is used with step sizes Δx and Δy;

[0167] Let the coordinates of the top left corner of the window be (x, y), then the window range is (x, y + W) × (y, y + H);

[0168] S423: Smoothness calculation:

[0169] For each window, calculate the mean smoothness of all points within the window:

[0170]

[0171] Where: N: the number of points within the window;

[0172] Window(x,y): Represents the set of points within the window area;

[0173] S424: Select the best matching position:

[0174] Based on the smoothness value within the sliding window, the region with the highest smoothness is selected as the optimal matching position:

[0175]

[0176] S5, the position and posture of the laser marking equipment are adjusted by the robotic arm to perform laser marking operation at the optimal marking position;

[0177] Specifically, in this embodiment, S5 is as follows:

[0178] The position of the laser marking equipment is adjusted by the robotic arm so that the laser marking equipment is aligned with the optimal marking position and perpendicular to the end face of the steel pipe to be laser marked, and the laser marking operation is performed at the optimal marking position.

[0179] In summary, this embodiment provides a method for determining and locating the position of steel pipe markings based on 3D vision technology. This method utilizes high-precision 3D vision equipment to acquire three-dimensional detail information of the steel pipe end. Through advanced image processing algorithms and positioning technology, it adaptively adjusts the equipment position and orientation to ensure the laser equipment is always perpendicular to the end face, identifies end face defects, and selects the optimal laser marking position on the end face that meets the marking conditions based on the length of the marking content issued by the system. This solves the problems of arbitrary position selection and poor marking effect during laser marking, and can meet the marking needs of steel pipes of various sizes and types. It improves the overall marking efficiency of the industry and ensures the accuracy and recognition rate of information.

[0180] Second Embodiment

[0181] This embodiment provides an electronic device, such as... Figure 3 As shown, the electronic device includes a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment described above. Furthermore, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0182] Below, in conjunction with Figure 3 A detailed introduction to each component of this electronic device is provided below:

[0183] The processor is the control center of the electronic device. The electronic device may include multiple processors, each of which can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The term "processor" can refer to a single processor or a collective term for multiple processing elements. For example, a processor can be one or more central processing units (CPUs), other general-purpose processors, application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), one or more field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor can perform various functions of the electronic device by running or executing software programs stored in memory and by calling data stored in memory.

[0184] In a specific implementation, as one example, the processor may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 shown are, of course, merely illustrative examples.

[0185] The memory is used to store the software program that executes the solution of the present invention, and the processor controls its execution. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.

[0186] Optionally, the memory may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may be integrated with the processor or may exist independently, and may be accessed through the interface circuit of the electronic device (…). Figure 3 (Not shown in the image) is coupled to the processor; however, this embodiment of the invention does not impose specific limitations on this.

[0187] The transceiver may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function. The transceiver can be integrated with the processor or exist independently, and is connected through the interface circuit of the electronic device (…). Figure 3 (Not shown in the image) is coupled to the processor, and this embodiment of the invention does not specifically limit this.

[0188] In addition, it should be noted that, Figure 3 The structure of the electronic device shown is not intended to limit the device. Actual devices may include more or fewer components than shown, or combine certain components, or have different component arrangements. Furthermore, the technical effects achieved by this electronic device when performing the method of the first embodiment described above can be referenced to the technical effects described in the first embodiment; therefore, they will not be repeated here.

[0189] Third Embodiment

[0190] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc. The instruction stored therein can be loaded and executed by a processor in a terminal.

[0191] Furthermore, it should be noted that the present invention can be provided as a method, apparatus, or computer program product. Therefore, embodiments of the present invention can take the form of a completely or partially hardware embodiment, a completely or partially software embodiment, or an embodiment combining software and hardware aspects. Moreover, when implemented in software, embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any usable medium accessible to a computer or a data storage device such as a server or data center containing one or more sets of usable media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive (SSD).

[0192] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0193] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0194] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element. Furthermore, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Additionally, the character " / " in this text generally indicates an "or" relationship between the preceding and following objects, but it can also indicate an "AND / OR" relationship. Please refer to the context for specific interpretations. "At least one" refers to one or more items, while "more than" refers to two or more items. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can be represented as: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0195] Furthermore, it is understood that in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0196] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0197] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of functional modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Additionally, the functional units in the various embodiments of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0198] If the method is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0199] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments of the present invention have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make several improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A method for determining and locating the position of steel pipe markings based on 3D vision technology, characterized in that, include: Using a robotic arm that integrates 3D vision equipment and laser marking equipment, the end face of the steel pipe within the field of view of the 3D vision equipment is 3D scanned at the initial position to obtain the first point cloud data; Based on the first point cloud data, the point cloud data corresponding to a single steel pipe end face is extracted, and the tilt angle of the steel pipe end face to be laser marked is calculated. According to the calculated tilt angle of the steel pipe end face, the 3D vision device is adjusted to the best shooting position by the robotic arm, and the 3D vision device after the position adjustment is used to perform 3D scanning on the steel pipe end face to be laser marked again to obtain the second point cloud data. Based on the second point cloud data, defects on the end face of the steel pipe are detected; Based on the defect detection results and the specifications of the content to be marked, the optimal marking position is selected on the end face of the steel pipe. The position of the laser marking equipment is adjusted by a robotic arm to perform laser marking at the optimal marking position. The step of extracting point cloud data corresponding to a single steel pipe end face based on the first point cloud data and calculating the tilt angle of the steel pipe end face to be laser marked includes: The first point cloud data is preprocessed; wherein, the preprocessing includes: firstly, using voxel grid downsampling to reduce the amount of point cloud data; and then removing noise points by statistical outlier removal. The preprocessed first point cloud data is divided into multiple clusters using the DBSCAN point cloud clustering algorithm, where each cluster represents the point cloud data corresponding to a steel pipe end face. For each cluster, perform plane fitting to obtain the equation and normal vector of the plane corresponding to the end face of the steel pipe; Based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser-marked, the tilt angle of the end face of the steel pipe to be laser-marked is calculated. Based on the defect detection results and the specifications of the content to be marked, the optimal marking position is selected on the end face of the steel pipe, including: Calculate the local smoothness of each point in the second point cloud data; The size of the sliding window is determined according to the specifications of the content to be identified. The sliding window is used to search the second point cloud data. The mean of the local smoothness of all points in each window is calculated as the smoothness value of the corresponding area in the sliding window. The area with the highest smoothness value and no defects is selected as the best identification position. The calculation of the local smoothness of each point in the second point cloud data includes: Calculate the normal vector of each point in the second point cloud data, including: Let p be the i-th point in the second point cloud data. i Find p i The set of neighborhood points N(p) i ); where the number of points in the neighborhood is k; Calculate the covariance matrix C of the neighborhood points i : Where, p j N(p) i The j-th point within ); μ i The mean of the neighborhood points is represented by T, which represents the transpose of the matrix. For C i Perform eigenvalue decomposition and take the eigenvector corresponding to the smallest eigenvalue as p. i normal vector n i ; Calculate the local smoothness at each point, including: For p i Calculate N(p) i The normal vectors of each point within n) and n i The differences between them; With N(p i The normal vectors of each point within n) and n i The mean of the differences between them is taken as p. i Local smoothness.

2. The method for determining and locating the position of steel pipe markings based on 3D vision technology as described in claim 1, characterized in that, The calculation of the tilt angle of the steel pipe end face to be laser-marked, based on the normal vector of the plane corresponding to the end face of the steel pipe to be laser-marked, includes: Calculate the angle θ between the normal vector of the plane corresponding to the end face of the steel pipe to be laser marked and the Z-axis direction vector; Calculate the tilt angle of the steel pipe end face to be laser marked.

3. The method for determining and locating the position of steel pipe markings based on 3D vision technology as described in claim 1, characterized in that, The detection of defects on the end face of the steel pipe based on the second point cloud data includes: The second point cloud data is preprocessed; the preprocessing includes normalization, noise reduction and downsampling operations. For the preprocessed second point cloud data, the improved PointNet++ deep learning algorithm is used to identify the size of the defects on the steel pipe end face and to locate the defects.

4. The method for determining and locating the position of steel pipe markings based on 3D vision technology as described in claim 3, characterized in that, The process involves using the improved PointNet++ deep learning algorithm to identify the size of defects on the steel pipe end face and locate the defects, based on the preprocessed second point cloud data. The PointNet++ algorithm is used to obtain the global features of the preprocessed second point cloud data. The global features of the preprocessed second point cloud data are compared with the features of the normal point cloud. The difference is calculated, and features with a difference greater than a preset threshold are mapped back to the original point cloud to determine the defect location. The size of the defect is estimated by analyzing the point cloud density and distribution in the defect area.

5. The method for determining and locating the position of steel pipe markings based on 3D vision technology as described in claim 1, characterized in that, The process of adjusting the position of the laser marking equipment using a robotic arm to perform laser marking at the optimal marking location includes: The position of the laser marking equipment is adjusted by the robotic arm so that the laser marking equipment is aligned with the optimal marking position and perpendicular to the end face of the steel pipe to be laser marked, and the laser marking operation is performed at the optimal marking position.

Images