Industrial pipeline-oriented robot control method and system

Abstract

The application discloses a kind of industrial pipeline-oriented robot control method and system, it is related to robot control technical field.The method includes: flange is installed on robot end effector, flange's front end center position installs binocular vision camera, four point laser displacement sensors are arranged on flange with different high symmetries;Binocular vision camera acquires pipeline image;Host computer determines the initial pose control result of robot according to pipeline image, to control robot to carry out preliminary movement;Four point laser displacement sensors are used to collect the spatial displacement data of pipeline at different height points after robot preliminary movement;Host computer determines the target pose control result of robot end effector by spatial displacement data, to control robot end effector to carry out attitude adjustment.The method can realize the accurate control of robot end effector and pipeline axial parallelism, to improve the accuracy of pipeline assembly.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular relates to a robot control method and system for industrial pipelines. BACKGROUND

[0002] It has become a widely concerned and significant thing to realize the automatic operation scene of industrial pipeline by a robot, and is gradually developing from emerging technology application to wide popularization.

[0003] In the scene of curved surface pipeline, the robot needs to align the end effector (such as a claw disc) to the end face of the cylindrical pipeline for assembly, butt joint or welding operation. In the prior art, the pose of the robot end effector is usually measured to realize the path control and position calibration of the robot end effector.

[0004] However, in the butt joint process of the curved surface object such as a pipeline, the prior art fails to effectively solve the problem of accurate control of the axial parallelism between the robot end effector and the pipeline, resulting in errors in pipeline assembly. SUMMARY

[0005] Therefore, it is necessary to provide a robot control method and system for industrial pipelines, which can realize accurate control of the axial parallelism between the robot end effector and the pipeline, thereby improving the accuracy of pipeline assembly.

[0006] The present application adopts the following technical solutions:

[0007] The present application provides a robot control method for industrial pipelines, a flange disc is installed on the end effector of the robot, a binocular vision camera is installed at the front end center position of the flange disc, and four point laser displacement sensors are arranged in a high-symmetry manner on the flange disc; the method comprises:

[0008] acquiring a pipeline image of the industrial pipeline by the binocular vision camera;

[0009] determining the pipeline position and the angle between the robot and the industrial pipeline according to the pipeline image, and determining the initial pose control result of the robot according to the pipeline position and the angle between the robot and the industrial pipeline, to control the robot to perform preliminary motion;

[0010] acquiring the space displacement data of the industrial pipeline at different height points by the four point laser displacement sensors after the robot performs preliminary motion;

[0011] The spatial displacement data is used to determine the angular deviation and positional offset between the robot end effector and the axis of the industrial pipeline, and a target pose control result of the robot end effector is determined according to the angular deviation and positional offset between the robot end effector and the axis of the industrial pipeline, so as to control the robot end effector to perform pose adjustment.

[0012] Optionally, the pipeline image includes a left image frame and a right image frame, and the pipeline position and the angle between the robot and the industrial pipeline are determined according to the pipeline image, including:

[0013] The left image frame and the right image frame are input into an end-to-end depth network PSMNet to obtain a pixel-level disparity map;

[0014] A depth map is obtained according to the pixel-level disparity map, the focal length and the baseline length of the binocular vision camera;

[0015] The depth map is input into an image semantic segmentation model to obtain a pipeline target bounding box; the image semantic segmentation model is constructed according to a lightweight network YOLOv8-Seg;

[0016] The pipeline position is determined according to the pipeline target bounding box;

[0017] The three-dimensional coordinates of the pipeline center point are calculated according to the intrinsic matrix of the binocular vision camera, the pipeline position and the corresponding disparity value thereof;

[0018] The angle between the robot and the industrial pipeline is determined according to the three-dimensional coordinates of the pipeline center point.

[0019] Optionally, the pipeline position includes the three-dimensional coordinates of the pipeline center point; an initial pose control result of the robot is determined according to the pipeline position and the angle between the robot and the industrial pipeline, including:

[0020] The robot base coordinate system and the pipeline coordinate system are aligned based on the three-dimensional coordinates of the pipeline center point and the angle between the robot and the industrial pipeline, to generate a straight-line motion path parameter from the current position of the robot to the industrial pipeline; the path parameter includes a translation amount and a rotation amount of the target pose of the end effector;

[0021] The translation amount and the rotation amount of the target pose are converted into angle instructions of each joint of the end effector to form the initial pose control result.

[0022] Optionally, the spatial displacement data includes the position coordinates of four spatial points; the angular deviation and positional offset between the robot end effector and the axis of the industrial pipeline are determined through the spatial displacement data, including:

[0023] The position coordinates of the four spatial points are fitted by using the least square method to obtain a normal vector of the pipeline axis;

[0024] determine an angular deviation between the robot end effector and the industrial pipeline axis according to a direction vector of the robot end effector and a normal vector of the industrial pipeline axis;

[0025] determine a position offset between the robot end effector and the industrial pipeline axis according to the coordinate of the center point of the pipeline and the coordinate point of the robot end effector.

[0026] Optionally, the position coordinates of the four spatial points are fitted by using a least square method to obtain the normal vector of the industrial pipeline axis, including:

[0027] construct a target function according to a three-dimensional space plane equation;

[0028] determine the value of the plane parameter in the target function according to the position coordinates of the four spatial points by using a least square method to solve the target function;

[0029] determine the value of the plane parameter in the target function as the normal vector of the industrial pipeline axis.

[0030] Optionally, the target function is:

[0031] ;

[0032] wherein, (P1, P2, P3, P4) represents the position coordinates of the i-th spatial point, , , P1, P2, P3, P4 are all variables. i , , and are all variables.

[0033] Optionally, the calculation formula of the angular deviation between the robot end effector and the pipeline axis is:

[0034] ;

[0035] wherein, represents the normal vector of the industrial pipeline axis, represents the direction vector of the robot end effector.

[0036] Optionally, the target pose control result of the robot end effector is determined according to the angular deviation and the position offset between the robot end effector and the industrial pipeline axis, including:

[0037] the joint angle that the robot end effector needs to adjust is determined by using an inverse kinematics algorithm according to the angular deviation and the position offset between the robot end effector and the industrial pipeline axis.

[0038] ​The application provides an industrial pipeline-oriented robot control system, which comprises a robot and an upper computer.

[0039] A binocular vision camera is used to collect a pipeline image of the industrial pipeline and send the pipeline image to the upper computer.

[0040] The upper computer is used to determine a pipeline position and an angle between the robot and the industrial pipeline according to the pipeline image, and determine an initial pose control result of the robot according to the pipeline position and the angle between the robot and the industrial pipeline, so as to control the robot to perform preliminary movement.

[0041] Four point laser displacement sensors are used to collect spatial displacement data of the industrial pipeline at different height points after the robot performs preliminary movement, and send the spatial displacement data to the upper computer.

[0042] The upper computer is used to determine an angle deviation and a position offset between the robot end effector and the industrial pipeline axis according to the spatial displacement data, and determine a target pose control result of the robot end effector according to the angle deviation and the position offset between the robot end effector and the industrial pipeline axis, so as to control the robot end effector to perform pose adjustment.

[0043] Optionally, the system further comprises a programmable logic controller, the upper computer communicates with the programmable logic controller, and the execution mechanism of the robot communicates with the programmable logic controller at high speed in a bidirectional mode.

[0044] The programmable logic controller is used to receive the pipeline image sent by the binocular vision camera and upload the pipeline image to the upper computer, receive the initial pose control result sent by the upper computer and send the initial pose control result to the execution mechanism of the robot, receive the spatial displacement data sent by the four point laser displacement sensors and upload the spatial displacement data to the upper computer, and receive the target pose control result sent by the upper computer and send the target pose control result to the execution mechanism of the robot.

[0045] The application provides a computer readable storage medium, the storage medium stores a computer program, and the computer program is executed by a processor to realize the above-mentioned XX method.

[0046] The application provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor realizes the above-mentioned XX method when executing the program.

[0047] The above-mentioned at least one technical scheme adopted by the application can achieve the following beneficial effects:

[0048] In the present application, first, a binocular vision camera is integrated on the flange plate, and then the initial motion of the robot is controlled based on the pipeline image collected by the binocular vision camera, so as to realize the rapid approach of the robot to the target pipeline. Four point laser displacement sensors are fixed in the form of different high symmetry on both sides of the end effector. When the robot reaches the vicinity of the pipeline, the spatial displacement data of the upper and lower points at both ends of the pipeline are collected respectively, and through the calculation of the angle deviation and position offset between the end effector of the robot and the axis of the pipeline, the posture adjustment of the end effector of the robot is realized, so as to realize the precise parallel docking of the end effector and the pipeline axis. In this way, the binocular vision camera and the point laser displacement sensor form a closed-loop cooperative mechanism of "rough positioning + precise positioning", so as to accurately realize the parallel control of the end effector of the robot and the pipeline axis, thereby improving the accuracy of the pipeline assembly. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings, which are included to provide a further understanding of the present application, form a part of the present application and illustrate embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0050] Figure 1 A structural schematic diagram of a robot control system for industrial pipelines provided by the present application;

[0051] Figure 2 A structural schematic diagram of a robot control system for industrial pipelines provided by the present application;

[0052] Figure 3 A flowchart of a robot control method for industrial pipelines provided by the present application;

[0053] Figure 4 A network structure diagram of YOLOv8-Seg. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in conjunction with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0055] In the scenarios of automated assembly, welding and handling of industrial pipes, robots need to achieve high-precision docking between the end effector (such as a claw disc or a gripper) and the pipe end face, and ensure good coaxial parallelism in the axial direction to avoid skewing, assembly failure or stress concentration. However, when facing challenges such as complex working conditions, limited space and uncertain pipe posture, traditional technologies have significant shortcomings in positioning accuracy, response speed and environmental adaptability.

[0056] 1. Limitations of existing technologies

[0057] (1) Systems that rely solely on robot body posture compensation ignore target relative posture control.

[0058] Most existing methods only achieve precision improvement by measuring and compensating the pose of the robot end effector, ignoring the spatial changes of external targets (such as pipes), making it difficult to meet the demand for target relative posture control in actual assembly. That is, most existing technologies focus on calibrating and compensating the posture of the robot itself through sensors to solve the motion accuracy problem of the robot. However, these methods often only optimize the path control and position calibration of the robot end effector, ignoring the relative posture control between the robot and external targets (such as pipes). Especially in the docking process of curved objects such as pipes, traditional technologies have not effectively solved the precise control of the axial parallelism between the robot end effector and the target pipe, resulting in assembly errors and insufficient execution precision.

[0059] (2) Vision positioning systems based on planar calibration boards cannot adapt to complex curved targets.

[0060] Many existing positioning technologies use standard planar calibration boards or external vision systems for positioning and pose measurement. This approach is effective for regular planes, but for axisymmetric, curved surface structures such as pipes, the calibration board cannot provide an effective reference datum, resulting in a significant decrease in positioning accuracy. Especially in complex industrial environments, it cannot adapt to frequent size changes, irregular targets, has poor deployment flexibility and unstable precision.

[0061] (3) Single-point vision or laser ranging methods have insufficient information dimension, i.e., insufficient sensor arrangement and error compensation mechanism, making it difficult to cope with complex working conditions.

[0062] Single visual recognition or laser point measurement system is subject to environmental lighting, shielding and surface reflection, and the measurement data lacks complete spatial structure information, making it difficult to support closed-loop adjustment between robots and targets in dynamic environments. And single-point or small-point measurement limits their positioning accuracy on complex curved surface objects (such as pipes). At the same time, error compensation algorithms are mostly based on simplified models, failing to fully utilize the spatial information of multi-point measurement data, resulting in low error compensation accuracy and failing to meet the requirements of high-precision dynamic positioning, especially in the precise docking task between robots and pipes.

[0063] 2、Industrial site core needs

[0064] The industrial site urgently needs an integrated and modular positioning system with the following functions:

[0065] (1) It can recognize pipes of various sizes and different attitudes, complete rapid coarse positioning and accurate docking;

[0066] (2) It can break away from the dependence on calibration plates and stable geometric reference, and adapt to irregular curved surface targets;

[0067] (3) It can realize the whole process closed-loop control from preliminary perception to attitude adjustment;

[0068] (4) It has the performance characteristics of anti-interference, high robustness and strong real-time performance.

[0069] Based on this, the present application provides a robot control method and system for industrial pipes, which is suitable for the automation operation scene of pipe type targets. By integrating binocular camera and laser sensor technology, it not only solves the problems of low positioning accuracy of robots in complex environments and low accuracy of parallelism between robot end effector and pipe axis in existing technology, but also solves the problems of slow response speed and poor system adaptability, specifically including the following aspects:

[0070] 1、The problem of poor positioning stability and insufficient accuracy of robots in complex environments

[0071] Traditional positioning methods often rely on manual vision or single sensor means. In the face of actual working conditions such as limited space, non-fixed pipe position or surface reflection, recognition failure or large error often occurs, making it difficult to meet the dual requirements of stability and accuracy in industrial sites. The present application integrates a vision camera on the robot flange to obtain the approximate spatial position of the pipe, realizes coarse positioning, and then combines four laser sensors to complete fine tuning, constructing a multi-modal perception collaborative link of "coarse positioning + precise positioning", thereby improving the positioning robustness and accuracy in complex environments.

[0072] 2、The problem of high-precision axial parallel positioning for curved pipes is solved accurately, and the problem of controlling the axial parallelism between the robot end effector and the pipe is solved accurately.

[0073] In practical applications, the robot needs to align the end effector (such as a claw disc) to the end face of the cylindrical pipe for assembly, butt joint or welding work. Since the outer surface of the pipe is a typical curved surface structure, and its installation attitude has uncertainty, it is a great challenge to achieve accurate alignment of the robot end with the pipe axis. The invention is different from the traditional technology of "robot body pose error compensation", focusing on the attitude relationship control between the robot end and the external target (pipe), rather than the traditional optimization of robot body error compensation, avoiding the dependence on robot kinematics modeling error and encoder accuracy from the system level, improving the universality and accuracy of axial parallel control. Specifically, by combining binocular camera and point laser displacement sensor, accurate control of the axial parallelism of the pipe and the robot end effector is realized, so that the robot can maintain high-precision parallel butt joint in complex curved surface workpiece (such as pipe) butt joint task, solving the precision problem that the existing technology cannot effectively deal with.

[0074] 3. No need to use external calibration board to realize space attitude solution, i.e. avoiding the use of calibration board to realize non-calibration arrangement.

[0075] Most of the current curved surface target positioning methods rely on standard plates or calibration blocks to construct spatial reference, which is limited by environmental space and target change frequency, and has complex on-site deployment and insufficient flexibility. The invention directly measures the relative height difference of the outer surface of the pipe at different positions through four symmetrically arranged point laser displacement sensors, establishes a spatial reference plane, and realizes the autonomous solution of the relative position and attitude between the pipe and the robot through real-time measurement and data fusion of the point laser displacement sensor, without the need for external calibration objects or additional visual markers, significantly improving the deployment flexibility and adaptability of the system, especially in on-site operations, rapid deployment and adjustment are possible.

[0076] 4. Sensor structure adaptability and curved surface target recognition robustness

[0077] Compared with the existing "coplanar" or "single-sided arrangement" sensor design, the four point laser displacement sensors are installed on the robot end in a structure of two-sided symmetry and different height from front to back, so that they can measure the spatial height information of the corresponding points on the upper and lower ends of the pipe, respectively. Through accurate multi-point measurement and advanced error compensation algorithm, the attitude model of the target is constructed. This arrangement structure has good self-adaptability to pipes of different diameters, materials, surface characteristics and installation angles, significantly improving the recognition accuracy and system robustness. The invention can accurately position the curved surface target such as the pipe, and adapt to the attitude changes of the pipe in different working states in real time. Through data fusion and dynamic error compensation, the robot end effector can always maintain accurate positioning and correct attitude under dynamic working conditions.

[0078] 5、Robot executor control closed loop response lag problem

[0079] In the traditional system, sensor data often needs to be analyzed manually or offline, and then the control instructions are issued by the operator, which has large feedback delay and low response efficiency. The application realizes real-time transmission and fusion processing of laser and visual sensor data by constructing a Profinet high-speed communication link between the PLC controller and the robot controller, and dynamically corrects and controls the closed loop based on the attitude error, effectively eliminates the system response lag, and improves the dynamic tracking capability in the pipeline positioning process.

[0080] 6、The adaptability of the system to the size and attitude change of the pipeline is insufficient

[0081] In actual industrial scenes, the attitude of the pipeline often changes due to process errors, thermal deformation, installation deviations and other factors, and the traditional system relies on fixed clamps or manual calibration, and lacks dynamic perception and adaptive adjustment capability. The application constructs a spatial reference plane in real time based on multi-point laser ranging data and attitude solving algorithm, and adjusts the attitude of the robot end executor through inverse kinematics, realizes rapid alignment and adaptive assembly of non-standard pipeline state, and has good industrial field adaptability.

[0082] The technical solutions provided by the embodiments of the application will be described in detail below with reference to the drawings.

[0083] Figure 1 For the implementation flowchart of a robot control system for industrial pipelines in the application, the system includes: a robot and an upper computer; a flange plate is installed on the robot end executor, a binocular vision camera is installed at the front end center position of the flange plate, and four point laser displacement sensors are arranged in a high asymmetric manner on the flange plate, and the four point laser displacement sensors are located on both sides of the robot end executor.

[0084] The binocular vision camera is used to collect the pipeline image of the industrial pipeline and send the pipeline image to the upper computer.

[0085] The upper computer is used to determine the pipeline position and the angle between the robot and the industrial pipeline according to the pipeline image, and determine the initial pose control result of the robot according to the pipeline position and the angle between the robot and the industrial pipeline, so as to control the robot to perform preliminary motion.

[0086] The four point laser displacement sensors are used to collect the spatial displacement data of the industrial pipeline at different height points after the robot performs preliminary motion, and send the spatial displacement data to the upper computer.

[0087] The host computer is configured to determine an angular deviation and a positional offset between the robot end effector and the industrial pipeline axis by using the spatial displacement data, and determine a target pose control result of the robot end effector according to the angular deviation and the positional offset between the robot end effector and the industrial pipeline axis, so as to control the robot end effector to perform pose adjustment.

[0088] The flange is directly installed on the robot end effector (tool interface of the mechanical arm end sixth axis) as a sensor integrated installation platform. Here, the "end" specifically refers to the physical terminal of the robot end effector, that is, the mechanical interface position directly interacting with the target object (pipeline) when the robot completes the work action. The flange is rigidly connected with the end effector, ensuring that the sensor module is completely synchronized with the robot end motion.

[0089] The installation position of the binocular vision camera: the binocular vision camera is fixed at the front end center position of the flange (facing the pipeline direction) to ensure that its field of view covers the pipeline target area. The camera optical axis is parallel to the default work direction of the robot end effector.

[0090] The system further comprises a programmable logic controller (PLC), the host computer communicates with the programmable logic controller, and high-speed bidirectional communication is performed between the robot and the programmable logic controller; the programmable logic controller is configured to receive the pipeline image sent by the binocular vision camera and upload the pipeline image to the host computer, receive the initial pose control result sent by the host computer and send the initial pose control result to the robot, receive the spatial displacement data sent by the four-point laser displacement sensor and upload the spatial displacement data to the host computer, and receive the target pose control result sent by the host computer and send the target pose control result to the robot.

[0091] The robot is a six-axis industrial robot.

[0092] The overall architecture of the system is shown in Figure 2 from left to right, and is composed of a host computer, a PLC, a robot, a binocular vision system, a laser sensor module and a positioning algorithm. The system constructs an intelligent positioning system with deep fusion of perception and control through five logical paths of perception-transmission-computation-control-execution.

[0093] The system has two main cooperative paths: the upper path (coarse positioning path): the binocular vision system and the robot cooperate to identify the approximate spatial position of the pipeline and output an initial pose estimation result; the lower path (fine positioning path): the point laser displacement sensor carried by the robot end flange and the robot control system cooperate to collect multi-point spatial distances and realize high-precision axis recognition and pose fine adjustment.

[0094] In Figure 2In the system, the modules are connected through seven key interaction points, which are as follows:

[0095] Label 1: The host computer communicates with the PLC through the network port to realize system parameter setting, instruction issuing and state monitoring.

[0096] Label 2: The PLC establishes real-time two-way communication with the robot through the Profinet high-speed industrial protocol to transmit positioning data and control instructions.

[0097] Label 3: The robot and the binocular vision system establish a coordinate transformation relationship (Robot-Vision Calibration) to realize accurate mapping of the vision coordinate to the robot coordinate system.

[0098] Label 4: The robot end flange is used as an execution platform and four point laser displacement sensors are installed to form a precise positioning execution foundation.

[0099] Label 5: The binocular vision system is responsible for initial recognition and coarse positioning, and extracts the approximate position, direction and topographic features of the pipeline through AI algorithms.

[0100] Label 6: The point laser displacement sensor adopts asymmetric arrangement to measure the distance of multiple key positions on the upper and lower ends of the pipeline in real time.

[0101] Label 7: Based on the laser sensor data, the axis positioning and attitude fitting algorithm is run to output high-precision axis pose solution results for guiding the robot to adjust the end attitude.

[0102] The system module composition specifically includes:

[0103] Host computer (human-machine interface and system scheduling center): The host computer communicates with the PLC through industrial Ethernet to realize process logic management, instruction issuing, state monitoring and data recording, and is the human-machine interaction and scheduling control center of the system.

[0104] PLC (logic control and communication hub): Using Profinet high-speed communication protocol, the PLC serves as the hub of information and instructions, responsible for connecting the sensing end and the execution end, real-time forwarding of binocular vision and laser sensor data, and controlling the robot action logic.

[0105] Robot (executive mechanism): As the core execution unit, the robot completes the alignment, approach and docking tasks of the pipeline target. Its end integrates a self-designed flange for carrying laser sensor modules and responding to the precise attitude adjustment instructions output by the sensing system.

[0106] Binocular vision system (coarse positioning perception mode): as a global perception mode, the binocular vision system captures large-scale position information of the pipeline, processes depth images through AI recognition algorithms, realizes preliminary perception of the pipeline contour, center axis and spatial pose, and provides coarse positioning guidance for the robot.

[0107] AI pipeline recognition algorithm (visual semantic modeling): based on visual image acquisition, combined with deep neural network algorithm, the pipeline structure is quickly recognized and reconstructed in space, and three-dimensional coordinate information is output X , Y , Z ) for the robot to guide the pose and perform preliminary insertion operation.

[0108] Self-designed end flange (multi-sensor integrated platform): the flange is used as an end device to fix and deploy four point laser displacement sensors arranged in different heights, and to ensure stable sampling and multi-point data redundancy acquisition, which is the hardware basis for realizing multi-modal fine perception.

[0109] Point laser displacement sensor array (fine positioning perception mode): four laser sensors are arranged in an array with different heights around the pipeline to obtain spatial displacement data of different height points of the pipeline, construct a multi-point three-dimensional geometric perception model, and realize high-precision fitting and pose refinement of the pipeline axis.

[0110] Axis positioning algorithm (space fitting and deviation solving engine): based on the multi-point data of the laser sensor, a space fitting model (least square plane, normal vector calculation, etc.) is constructed to real-time calculate the angle deviation and position offset between the robot end effector and the pipeline axis, and drive the robot end to perform dynamic adjustment.

[0111] The application also provides a robot control method for an industrial pipeline, which is applied to the above-mentioned system, as shown in Figure 3 The method comprises the following steps:

[0112] S301, acquiring a pipeline image of an industrial pipeline through a binocular vision camera; determining a pipeline position and an angle between the robot and the industrial pipeline according to the pipeline image, and determining an initial pose control result of the robot according to the pipeline position and the angle between the robot and the industrial pipeline, to control the robot to perform preliminary motion.

[0113] Optionally, the pipeline image comprises a left image frame and a right image frame, and the pipeline position and the angle between the robot and the industrial pipeline are determined according to the pipeline image, comprising: inputting the left image frame and the right image frame into an end-to-end depth network PSMNet to obtain a pixel-level disparity map; obtaining a depth map according to the pixel-level disparity map, the focal length and the baseline length of the binocular vision camera; inputting the depth map into an image semantic segmentation model to obtain a pipeline target bounding box; the image semantic segmentation model is constructed according to a lightweight network YOLOv8-Seg; determining the pipeline position according to the pipeline target bounding box; calculating the three-dimensional coordinates of the pipeline center point according to the intrinsic matrix of the binocular vision camera, the pipeline position and the corresponding disparity value; determining the angle between the robot and the industrial pipeline according to the three-dimensional coordinates of the pipeline center point.

[0114] Specifically, the method comprises the following steps:

[0115] (1) Image input and disparity calculation: left image frame: I_L( x , y ), right image frame: I_R( x , y ), camera parameters: focal length f , baseline length B and intrinsic matrix K .

[0116] The end-to-end depth network PSMNet calculates the pixel-level disparity map D( x , y ), and the depth map is calculated by using the binocular triangulation principle: .

[0117] (2) Pipeline target recognition: as shown in Figure 4 , Figure 4Figure 1 is a network structure diagram of YOLOv8-Seg, wherein YOLOv8-Seg comprises a backbone network and a head network, the CBS module in the figure comprises a convolution layer (Convolution), a batch normalization layer (Batch Normalization, BN) and a SiLU activation function connected in series: the C2f module is a cross-stage local fusion module. It divides the input features into two parts, one part is directly transmitted, and the other part is spliced and fused with the directly transmitted part after multiple convolution operations, thereby enhancing the feature extraction capability and reducing the calculation amount; the SPPF module is a spatial pyramid pooling fast (Spatial Pyramid Pooling Fast) module; Upsample represents the up-sampling operation, Concat represents the splicing operation, Segment represents the instance segmentation, and based on the extracted and fused feature map, the final instance segmentation result is output, for example, the category of the predicted target and the corresponding mask, so as to realize the accurate segmentation of different instances in the image. The depth map is input into the image semantic segmentation model to obtain the pipeline target bounding box bbox = [ x min , y min , x max , y max ] and the semantic mask M( x , y ).

[0118] Specifically, the semantic mask M( x , y ) is a binary matrix of the same size as the image, which marks whether each pixel belongs to the pipeline target (the pipeline region is 1 and the background is 0), realizing pixel-level pipeline contour segmentation.

[0119] The physical meaning of the semantic mask M( x , y ) represents the accurate pixel-level region of the pipeline target in the depth map, which is used to extract the pipeline edge contour point set for subsequent three-dimensional direction calculation (such as fitting the pipeline main axis direction by principal component analysis), and combined with the depth map Z( x , y ), the pixel of the pipeline region is mapped to a three-dimensional point cloud, which is used to calculate the pipeline center point coordinates ( X , Y , Z ).

[0120] YOLOv8-Seg generates M( x , yFeature extraction on input depth map, output bounding box and segmentation mask of pipeline target. The final semantic mask M is obtained by mask thresholding (such as Sigmoid activation + 0.5 binaryzation). x , y ).

[0121] (3) Three-dimensional coordinate recovery (image coordinates → space coordinates)

[0122] The pipeline position can include the center point of the image, and the image center point (x, y) is calculated x c , y c ): .

[0123] Three-dimensional coordinate recovery (image coordinates → space coordinates): according to the intrinsic matrix K of binocular vision camera, the image center point (x, y) and its corresponding disparity value D(x, y), the three-dimensional coordinates of the pipeline center point (X, Y, Z) are calculated x c , y c ) and , . Depth recovery: X , Y , Z ); pixel coordinate projection: intrinsic matrix K includes principal point (x0, y0) and pixel size ratio x , c y ) and c x and f y : f .

[0124] (4) Rough direction estimation

[0125] Extract edge contour points from mask M(x, y), and use linear fitting or principal component analysis (PCA) to get two-dimensional principal axis vector: .

[0126] Output the angle between the robot and the industrial pipeline: .

[0127] where is the unit vector of the two-dimensional principal axis vector, which describes the main extension direction of the pipeline in the image plane. Combined with depth information Z ( x , y ) in the future, it can be converted into a pipeline direction vector in three-dimensional space.

[0128] ​Specifically, The acquisition method comprises:

[0129] In one embodiment, the principal component analysis is used to acquire: input: pixel coordinate set of mask M( x , y )=1.

[0130] Step: calculate the covariance matrix of pixel coordinates, perform eigenvalue decomposition on the covariance matrix, and the eigenvector corresponding to the maximum eigenvalue is the principal axis direction (v , ).

[0131] In another embodiment, the linear fitting is used to acquire: input: pixel coordinate set of mask M( x , y )=1, and step: assuming that the pipe profile is approximately a straight line, a straight line equation is fitted using the least square method y = kx + b .

[0132] The principal axis direction vector is v=(1, k ), and the normalized vector is .

[0133] In one embodiment, the pipe position comprises three-dimensional coordinates of a pipe center point; and according to the pipe position and the angle between the robot and the industrial pipe, an initial pose control result of the robot is determined, comprising: aligning a robot base coordinate system with a pipe coordinate system based on the three-dimensional coordinates of the pipe center point and the angle between the robot and the industrial pipe, to generate a straight line motion path parameter from the current position of the robot to the industrial pipe; the path parameter comprises a translation amount and a rotation amount of a target pose of an end effector; and the translation amount and the rotation amount of the target pose are converted into angle instructions of each joint of the end effector to form the initial pose control result.

[0134] Specifically, based on the three-dimensional coordinates of the pipe center point ( X , Y , Z ) calculated by the binocular vision system and the angle θ between the robot and the pipe, the robot base coordinate system is aligned with the pipe coordinate system to generate a straight line motion path from the current position of the robot to the proximal rough positioning point of the pipe. The path parameter comprises a translation amount ΔP = [ X , Y , Z ]^T and a rotation amount ΔR = R_z(θ) of a target pose of an end effector, and the motion mode adopts joint space interpolation or Cartesian space straight line motion.

[0135] Initial pose control result generation: the target pose (ΔP, ΔR) is converted into joint angle instructions through robot forward kinematics to form the initial pose control result. The calculation formula is:

[0136]

[0137] where IK represents the inverse kinematics solution function, is the joint angle vector.

[0138] Safety constraints and collision detection: introduce robot workspace restrictions and joint speed / acceleration constraints in path planning to ensure collision-free and smooth motion during the coarse positioning process.

[0139] S302, after the robot performs preliminary motion, the spatial displacement data of the industrial pipeline at different height points is collected through the four-point laser displacement sensor.

[0140] S303, through the spatial displacement data, the angle deviation and position offset between the robot end effector and the industrial pipeline axis are determined, and the target pose control result of the robot end effector is determined according to the angle deviation and position offset between the robot end effector and the industrial pipeline axis, to control the robot end effector to perform attitude adjustment.

[0141] In one embodiment, the spatial displacement data includes the position coordinates of four spatial points; through the spatial displacement data, the angle deviation and position offset between the robot end effector and the industrial pipeline axis are determined, including: fitting the position coordinates of the four spatial points to obtain the normal vector of the industrial pipeline axis using the least squares method; according to the direction vector of the robot end effector and the normal vector of the industrial pipeline axis, the angle deviation between the robot end effector and the pipeline axis is determined; according to the center point coordinates of the pipeline and the coordinate point of the robot end effector, the position offset between the robot end effector and the industrial pipeline axis is determined.

[0142] Optionally, fitting the position coordinates of the four spatial points to obtain the normal vector of the industrial pipeline axis using the least squares method includes: constructing an objective function according to a three-dimensional space plane equation; solving the objective function using the least squares method to determine the value of the plane parameter in the objective function according to the position coordinates of the four spatial points; determining the value of the plane parameter in the objective function as the normal vector of the pipeline axis.

[0143] The three-dimensional space plane equation is: , A , B , C and D is the plane parameter, ( x , y , z) are three-dimensional plane coordinates.

[0144] The objective function is:

[0145] ;

[0146] wherein (P1 , , ) represents the position coordinates of the first spatial point, i , , and are all variables.

[0147] The point laser displacement sensor outputs the position coordinates of four spatial points, respectively: P1 x 1, y 1, z 1), P2 x 2, y 2, z 2), P3 x 3, y 3, z 3) and P4 x 4, y 4, z 4).

[0148] Based on the position coordinates of the four spatial points, the least squares method is used to fit the objective function to obtain the values of the plane parameters , , and , that is, the normal vector of the industrial pipeline axis = ( A , B , C ).

[0149] The direction vector of the robot's end effector is , and the included angle between the industrial pipeline axis and the robot's end effector is calculated by the following formula: .

[0150] Therefore, the calculation formula of the angle deviation between the robot's end effector and the industrial pipeline axis is:

[0151] ;

[0152] wherein represents the normal vector of the industrial pipeline axis, represents the direction vector of the robot's end effector.

[0153] ​Position offset between robot end effector and industrial pipe axis: P_center is the center point coordinate of the pipe ( , , ), P_effector is the coordinate of the robot end effector ( , , ), then the position offset ΔT is:

[0154] .

[0155] Wherein, the method for obtaining the pipe center point coordinate (P_center) is: using the YOLOv8-Seg model to obtain the semantic mask M(x, y), extracting the pixels of the pipe region, and averaging or weighted calculating the depth values (from the depth map Z(x, y)) of these pixels to obtain the three-dimensional center point, i.e. the center point coordinate of the pipe.

[0156] By applying the camera intrinsic matrix and coordinate conversion, the image coordinate system is converted to the robot base coordinate system, and the method for obtaining P_effector (the coordinate of the robot end effector) is: the real-time feedback of the end effector pose information of the robot controller.

[0157] In one embodiment, according to the angle deviation and position offset between the robot end effector and the axis of the industrial pipe, the target pose control result of the robot end effector is determined: according to the angle deviation and position offset between the robot end effector and the axis of the industrial pipe, the joint angle that the end effector of the robot needs to adjust is determined through inverse kinematics algorithm. By adjusting the joint angle of the robot end effector, it is ensured that the end effector and the pipe axis are kept in alignment.

[0158] In one embodiment, the point laser displacement sensor array and angle calculation

[0159] 1) Horizontal sensor calculation

[0160] Known:

[0161] The fixed distance between the horizontal sensors:

[0162] The distance value measured in the horizontal direction: and

[0163] The spatial coordinates of sensor 1 are: x 1, y 1, z 1)

[0164] The spatial coordinates of sensor 2 are: x 2,y 2, z 2)

[0165] Calculation steps:

[0166] Calculate the angle between the horizontal direction sensor and the pipe axis.

[0167] For each horizontal direction sensor, calculate the angle between it and the pipe axis. The direction of the pipe axis is n pipe = [ a , b , c ], calculate the angle θ h :

[0168]

[0169]

[0170] Coordinate transformation: use the spatial coordinates of the sensor and the target pipe axis to perform coordinate transformation. P Let the coordinates of a point on the pipe axis be T h , and the transformation matrix for the horizontal direction be

[0171] 2) Vertical direction sensor calculation

[0172] Known:

[0173] The fixed distance between the vertical direction sensors:

[0174] The distance value measured in the vertical direction: d 3 and d 4

[0175] The spatial coordinates of sensor 3 are: x 3, y 3, z 3)

[0176] The spatial coordinates of sensor 4 are: x 4, y 4, z 4)

[0177] Calculation steps:

[0178] Calculate the angle in the vertical direction:

[0179] For each vertical direction sensor, calculate the angle between it and the pipe axis θ v :

[0180]

[0181]

[0182] Note that the angle calculation of the horizontal and vertical directions ( θ h , θ v ) is a fine decomposition and dynamic compensation of the global angle deviation, which works as follows:

[0183] 1. Split-axis control

[0184] Horizontal angle θ h : Guide the robot to rotate around the Z axis to correct the deflection of the end effector in the horizontal plane.

[0185] Vertical angle θ v : Guide the robot to rotate around the X axis to correct the tilt of the end effector in the vertical plane.

[0186] Advantages: Decompose the comprehensive angle deviation into independent components to avoid adjustment oscillation caused by multi-joint coupled motion.

[0187] 2. High-frequency fine-tuning

[0188] The laser sensor updates at a frequency of 1 kHz θ h and θ v , real-time response to pipeline vibration or robot motion error, and realize dynamic closed-loop control.

[0189] 3. Associated process

[0190] Coarse positioning: vision calculates the global deviation and drives the robot to quickly approach the pipeline.

[0191] Fine positioning: laser calculates the direction θ h / θ v , to achieve sub-millimeter level posture locking.

[0192] In one embodiment, please continue to see Figure 2 , the system process follows the closed-loop control of the five logical paths of "perception - transmission - calculation - control - execution", the specific process is as follows:

[0193] Step 1: Initialize communication between host computer and PLC

[0194] The host computer issues task parameters, establishes connection with the PLC through Ethernet, configures the robot operation process, initializes the perception module and waits for response.

[0195] Step 2: Binocular vision system starts coarse positioning

[0196] The robot moves to the initial observation position, the binocular vision camera acquires the stereoscopic image data of the pipeline area, the AI recognition algorithm extracts the pipeline contour and axial features, and completes the initial positioning in space. Through the vision-robot calibration matrix, the coordinate system conversion is completed, and the robot is preliminarily guided to approach the target.

[0197] Step 3: Robot performs coarse pose adjustment

[0198] Based on the visual perception results, the robot performs coarse alignment action, moves the end effector (flange) to the proximal end of the pipeline, and completes the preparation for near-field docking.

[0199] Step 4: Laser sensor fine positioning starts

[0200] After the robot approaches the target, the four-point laser displacement sensor on the flange is turned on, and the accurate displacement difference of the upper and lower points on both ends of the pipeline is obtained, realizing fine modeling of the spatial pose of the pipeline.

[0201] Step 5: Axis positioning algorithm real-time fitting pipeline pose

[0202] The system performs spatial calculation on the four-point laser data, fits the pipeline axis, and real-time calculates the relative deviation (angle, distance, direction) between the robot end and the pipeline axis, forming a fine positioning feedback quantity.

[0203] Step 6: Robot inverse solution drives end fine adjustment

[0204] According to the deviation results, the robot runs the inverse kinematics algorithm to adjust the multi-axis pose, corrects the end effector, and makes it keep high-precision parallel with the pipeline axis, entering the precise insertion area.

[0205] Step 7: Closed-loop control and dynamic feedback correction

[0206] PLC and robot maintain high-speed Profinet data channel, continuously acquire laser sensing data, realize high-frequency closed-loop adjustment, adapt to pipeline dynamic micro-deviation and vibration, and realize truly multi-modal, continuous and dynamic pose adaptive control.

[0207] The robot control method for industrial pipelines constructed by the application forms a closed-loop cooperative mechanism of "coarse positioning + fine positioning" through a "staged, two-level" perception strategy:

[0208] (1) Binocular camera and robot cooperate to realize spatial coarse positioning

[0209] The application integrates binocular vision cameras in front of the robot flange, which is used to obtain the approximate spatial position and orientation of the pipeline. The pipeline center point and axial direction are calculated by a stereo vision algorithm to realize the fast approach of the robot to the target pipeline, build the positioning initial framework, and significantly improve the response speed and target capture efficiency of the robot.

[0210] (2) Laser sensor cooperates with robot to realize precise positioning of posture

[0211] After the robot reaches the vicinity of the pipeline, four point laser displacement sensors are fixed on both sides of the end effector in a symmetrical manner to collect the distance data of the upper and lower points on both ends of the pipeline. The accurate axial direction of the pipeline is calculated in real time by a four-point fitting algorithm, and the end effector is precisely parallel to the axial direction of the pipeline by combining the robot end pose solving system.

[0212] (3) Sensory fusion and closed-loop control mechanism

[0213] A Profinet high-speed communication channel is established between the PLC controller and the robot body control system to realize real-time fusion of binocular vision and laser ranging information. After data processing, the pose adjustment instructions of the robot end are converted, and the joint linkage is driven through inverse kinematics to realize closed-loop response and dynamic posture compensation.

[0214] The technical advantages and breakthroughs of the application include:

[0215] (1) Multi-modal perception structure is clear and function division is clear: binocular camera focuses on global positioning, laser sensor focuses on local fine adjustment, and system stability and adaptability are enhanced.

[0216] (2) Adapt to irregular targets: The system does not depend on regular planes or calibration devices and is suitable for industrial pipelines of different sizes, attitudes and materials.

[0217] (3) Easy to deploy and quickly reusable: The sensing module is integrated into the robot end, which can be used immediately after installation, and is flexible and suitable for various industrial sites.

[0218] (4) High robustness and real-time control: The response delay is eliminated through the closed-loop control system to realize real-time tracking and adjustment of the posture difference in dynamic scenes.

[0219] The key points of the application are:

[0220] (1) Multi-modal collaborative perception architecture for positioning tasks

[0221] The application designs a robot control method for industrial pipelines around the high-precision positioning task of pipeline axisymmetric targets. The method introduces a heterogeneous fusion mechanism of visual perception and laser displacement perception. Vision is responsible for macro target cognition, and laser is responsible for micro posture analysis and local error modeling, which realizes the whole chain closed loop of "task-oriented-multimodal perception-posture reasoning-control decision". By introducing visual sensing and laser displacement sensing means, target rough identification and micro posture modeling are completed respectively. Through data fusion, coordinate registration and error collaborative compensation mechanism, dynamic posture adjustment of the robot end effector is realized, which is suitable for cylindrical pipeline target docking scene. Specifically, first, the binocular vision camera locates the pipeline entrance area and identifies the initial value of the pipeline posture, guiding the laser mode to arrange the docking position; the laser sensor performs fine positioning with high frequency sampling, and completes robot posture solving and adjustment based on multimodal collaborative strategy.

[0222] (2) Rough and fine collaborative positioning strategy under heterogeneous sensor fusion

[0223] In the multimodal sensing structure, the vision system first completes rough identification and space estimation of the pipeline area, providing a preliminary search area for the laser subsystem. The four-point laser displacement sensor array performs micro high-frequency sampling based on rough positioning, and establishes the real axis model of the pipeline through spatial geometric solving. This strategy not only improves the positioning accuracy, but also significantly improves the adaptive ability of the robot in unstructured environments. Based on the fixed arrangement relationship between sensors and the difference in measurement data, this method directly constructs the relative spatial posture model of the target, avoiding the traditional coordinate conversion and calibration process, and improving the deployment flexibility and model generality.

[0224] (3) Structured space reasoning mechanism constructed by four-point heterogeneous laser array

[0225] By arranging four point laser displacement sensors symmetrically along the flange at the end of the robot gripper, a "heterogeneous non-coplanar measurement array" in space is constructed. Using the geometric difference of the measurement points, the three-dimensional spatial posture is reconstructed, and the active adjustment of the axial parallelism is supported. Four point laser displacement sensors are arranged asymmetrically along the flange edge of the pipeline, forming a stable non-coplanar measurement array; by constructing a spatial model based on the distance difference between the laser points on the two end surfaces and the pipeline surface, the fitting of the target axis and the real-time posture error feedback are realized.

[0226] (4) "Perception internal parameter autonomous mechanism" without external calibration reference

[0227] The application breaks the dependence of traditional robot systems on external calibration boards, global coordinate systems, etc., and proposes a "perception internal parameter autonomous" deployment method: this method is completely based on the relative structural relationship between sensors, and realizes pipeline space posture estimation through internal geometric modeling and data fitting, with the ability of rapid deployment and adaptation to different specifications. The real-time perception and calculation of the posture error parameters are used as the input of the robot inverse kinematics control, combined with the high-speed communication protocol between PLC and robot to realize high-frequency data updating, so as to implement dynamic posture compensation control on the robot end effector.

[0228] The robot control method for industrial pipelines provided by the application brings the following effects:

[0229] 1. Dual-mode collaborative perception improves positioning accuracy and spatial understanding ability

[0230] By combining the collaborative work of the binocular vision system and the four-point laser displacement sensor, the application realizes high-precision three-dimensional positioning of the pipeline target, with positioning accuracy improved to ±0.01mm. Compared with the traditional single sensor system (with positioning accuracy usually ±0.5mm), this significantly improved accuracy ensures the reliability and precision of the robot in high-precision assembly, pipeline docking, welding and other scenarios, effectively avoiding operation failure and quality problems caused by positioning errors.

[0231] 2. Real-time response and efficient dynamic control

[0232] The application realizes high-speed, low-latency data transmission between PLC and robot control system through Profinet protocol, with communication delay reduced to <5ms. This low-latency response significantly improves the dynamic response capability of the system, enabling the robot to quickly respond to changes in pipeline posture in high-speed production lines, accurately adjust the posture of the end effector, and ensure efficient and stable production operations.

[0233] 3. Enhance system stability and robustness

[0234] Through the closed-loop control system, the application can monitor laser sensor data in real time and automatically adjust the posture of the robot end effector, ensuring that the robot always maintains high-precision parallelism with the pipeline axis. This control system effectively suppresses errors caused by external interference, pipeline deformation, equipment deviation and other factors, improving system stability and robustness. Even under long-term high-load operation, the positioning accuracy remains ≥99.5%.

[0235] 4. Significantly improve production efficiency and reduce manual intervention

[0236] Compared with traditional manual calibration or low-precision automation systems, the robot can complete pipeline positioning and position calibration within a few seconds through real-time data feedback and closed-loop control, reducing the time consumption by >80% compared with traditional systems (which usually take several minutes).

[0237] 5. Improve system adaptability and compatibility

[0238] The system design of the present application has high adaptability and can support pipeline operations of different diameters, materials (such as steel pipes, plastic pipes, composite pipes, etc.), shapes (such as straight pipes, curved pipes), etc., ensuring the universality of the system in different industrial scenarios. Through modular design, the system can quickly adapt to different specifications of the pipeline target and can support a wider range of workpiece size changes, with an improvement of >50% in adaptability.

[0239] 6. Reduce system failure rate and maintenance cost

[0240] By using high-precision laser sensors and stable communication protocols, the system failure rate is reduced by about 70% compared with traditional mechanical touch or optical sensors. The real-time error compensation mechanism of the system reduces the positioning error and deviation of the robot, reduces frequent adjustment and maintenance work, reduces long-term maintenance cost, and greatly reduces the downtime of the equipment.

[0241] 7. Enhance the intelligent level and adaptive ability of the system

[0242] The present application integrates intelligent error compensation algorithms and dynamic feedback control mechanisms, enabling the robot to automatically adjust the operation path and correct the target attitude deviation in real time, reducing the dependence on manual operation. The robot can adaptively optimize the control strategy according to different working environments and continuously optimize itself according to feedback data, thereby realizing autonomous processing capability for complex tasks and further improving the intelligent level and production automation level of the system.

[0243] The specific limitations of the robot control system for industrial pipelines provided by the present application can be referred to the limitations of the robot control method for industrial pipelines described above, which will not be repeated here.

[0244] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

Claims

1. A robot control method for industrial pipelines, characterized in that, A flange is mounted on the robot's end effector. A binocular vision camera is mounted at the center of the front end of the flange. Four point laser displacement sensors are symmetrically arranged on the flange with varying heights. The method includes: Pipeline images of industrial pipelines are acquired using a binocular vision camera; The pipe position and the angle between the robot and the industrial pipe are determined based on the pipe image. The initial pose control result of the robot is determined based on the pipe position and the angle between the robot and the industrial pipe to control the robot to perform initial movements. After the robot performs its initial movements, spatial displacement data of the industrial pipeline at different heights is collected using four point laser displacement sensors; the spatial displacement data includes the position coordinates of the four spatial points. Based on the equations of the three-dimensional plane, construct the objective function; the objective function is: ； in,( , , ) indicates the first i The position coordinates of a point in space, , , and All are planar parameters; Based on the position coordinates of four spatial points, the objective function is solved using the least squares method to determine the values ​​of the plane parameters in the objective function; Based on the values ​​of the plane parameters in the objective function, determine the normal vector of the industrial pipeline axis. = ( A , B , C ); Based on the direction vector of the robot's end effector and the normal vector of the pipeline axis, determine the angular deviation between the robot's end effector and the industrial pipeline axis. Based on the coordinates of the center point of the pipeline and the coordinates of the robot's end effector, the positional offset between the robot's end effector and the axis of the industrial pipeline is determined. Based on the angular deviation and positional offset between the robot's end effector and the axis of the industrial pipeline, the target pose control result of the robot's end effector is determined to control the robot's end effector to adjust its posture.

2. The method according to claim 1, characterized in that, The pipeline image includes a left image frame and a right image frame. The pipeline position and the angle between the robot and the industrial pipeline are determined based on the pipeline image, including: The left and right image frames are input into the end-to-end deep network PSMNet to obtain a pixel-level disparity map. A depth map is obtained based on the pixel-level disparity map, the focal length of the binocular vision camera, and the baseline length. The depth map is input into the image semantic segmentation model to obtain the target bounding box of the pipeline; the image semantic segmentation model is built based on the lightweight network YOLOv8-Seg; Determine the pipeline location based on the pipeline target bounding box; Calculate the three-dimensional coordinates of the pipe center point based on the intrinsic parameter matrix of the binocular vision camera, the pipe position, and its corresponding disparity value; The angle between the robot and the industrial pipeline is determined based on the three-dimensional coordinates of the pipeline's center point.

3. The method according to claim 2, characterized in that, The pipeline position includes the three-dimensional coordinates of the pipeline's center point; based on the pipeline position and the angle between the robot and the industrial pipeline, the initial pose control results of the robot are determined, including: Based on the three-dimensional coordinates of the pipeline center point and the angle between the robot and the industrial pipeline, the robot's base coordinate system is aligned with the pipeline coordinate system to generate straight-line motion path parameters from the robot's current position to the industrial pipeline; the path parameters include the translation and rotation of the end effector's target pose. The translation and rotation of the target pose are converted into angle commands for each joint of the end effector to form the initial pose control result.

4. The method according to claim 1, characterized in that, Angular deviation between the robot end effector and the axis of the industrial pipeline The calculation formula is: ； in, The normal vector representing the axis of the industrial pipeline. This represents the direction vector of the robot's end effector.

5. The method according to claim 1, characterized in that, Based on the angular deviation and positional offset between the robot end effector and the axis of the industrial pipeline, the target pose control result of the robot end effector is determined, including: Based on the angular deviation and positional offset between the robot's end effector and the axis of the industrial pipeline, the joint angles that the robot's end effector needs to be adjusted are determined using an inverse kinematics algorithm.

6. A robot control system for industrial pipelines, characterized in that, The system is used to implement the method described in any one of claims 1-5, the system comprising: a robot and a host computer; a flange is installed on the robot's end effector, a binocular vision camera is installed at the center of the front end of the flange, and four point laser displacement sensors are arranged symmetrically on the flange with different heights; A binocular vision camera is used to acquire images of industrial pipelines and send these images to a host computer. The host computer is used to determine the position of the pipeline and the angle between the robot and the industrial pipeline based on the pipeline image, and to determine the initial pose control result of the robot based on the pipeline position and the angle between the robot and the industrial pipeline, so as to control the robot to perform initial movements. Four point laser displacement sensors are used to collect spatial displacement data of the industrial pipeline at different heights after the robot has made initial movements, and send the spatial displacement data to the host computer; the spatial displacement data includes the position coordinates of the four spatial points; The host computer is used to construct the objective function based on the three-dimensional spatial plane equation; the objective function is: ； in,( , , ) indicates the first i The position coordinates of a point in space, , , and All are planar parameters; Based on the coordinates of four spatial points, the objective function is solved using the least squares method to determine the values ​​of the plane parameters in the objective function; based on the values ​​of the plane parameters in the objective function, the normal vector of the industrial pipeline axis is determined. =( A , B , C Based on the direction vector of the robot's end effector and the normal vector of the pipeline axis, the angular deviation between the robot's end effector and the industrial pipeline axis is determined; based on the coordinates of the pipeline center point and the coordinates of the robot's end effector, the positional offset between the robot's end effector and the industrial pipeline axis is determined; and based on the angular deviation and positional offset between the robot's end effector and the industrial pipeline axis, the target pose control result of the robot's end effector is determined to control the robot's end effector to adjust its posture.

7. The system according to claim 6, characterized in that, The system also includes a programmable logic controller (PLC), with the host computer communicating with the PLC and the robot's actuators communicating with the PLC at high speed in both directions. The programmable logic controller (PLC) is used to receive pipeline images sent by the binocular vision camera and upload them to the host computer; receive the initial pose control results sent by the host computer and send them to the robot's actuators; receive spatial displacement data sent by four point laser displacement sensors and upload them to the host computer; and receive the target pose control results sent by the host computer and send them to the robot's actuators.

Images