Visual inspection method and device for straightness of pipe

Abstract

The invention discloses a visual detection method and device for the straightness of a pipe, and relates to the technical field of pipe straightness detection.The straightness of the pipe is detected in the process that the pipe moves forwards along the axis, two cameras with imaging planes perpendicular to each other are arranged on the circumference of the pipe, and the main optical axes of the two cameras are both perpendicular to the axial direction of the pipe; obtaining internal and external parameters of the two cameras through calibration and establishing a three-dimensional coordinate system; determining the image taking frequency of a camera according to the pipe length and the movement speed, so that an overlapping area exists between adjacent frames of the camera; the two cameras shoot two pipe images at the same time according to the image taking frequency, feature extraction is conducted on the two pipe images respectively to obtain the pixel coordinates of the central axis of the pipe, and a three-dimensional coordinate point set of the central axis of the current section of the pipe is obtained through calculation; calculating a splicing matrix between the adjacent frames according to an overlapping area between the adjacent frames, and splicing the three-dimensional coordinate point sets of the central axes of the multiple sections of pipes to obtain a three-dimensional coordinate point set of the central axis of the whole pipe; and performing straight line fitting on the three-dimensional coordinate point set of the central axis of the whole pipe, and evaluating the straightness of the pipe by using a straight line fitting error. The device has the advantages of being simple in structure, low in cost, capable of achieving online detection, high in precision, good in universality and the like, and can be rapidly applied to straightness detection of various pipes.

Description

Technical Field

[0001] The invention relates to the technical field of pipe straightness detection, in particular to a method and a device for visually detecting pipe straightness. Background Art

[0002] Tube materials are one of the most commonly used materials in industry. In the production process of various types of pipes, they undergo multiple complex processes such as cutting, grinding, extrusion, and heat treatment, which are prone to bending and deformation. If not handled in time, it will affect subsequent processing and use. With the improvement of quality requirements for pipes, straightness detection to measure pipe quality is of utmost importance. At present, straightness detection methods mainly include manual detection, laser detection, and three-coordinate method. Manual detection can only be performed offline, using conventional feeler gauges or molds for matching, and it is impossible to perform online detection on longer pipes; the laser collimator method is easily affected by external interference and is affected by the material of the pipe, so the accuracy is unstable and is not suitable for general production and processing; although the three-coordinate method has high accuracy, the cost is too high, and the detection method can only be point-to-point contact measurement, which is inefficient. Summary of the invention

[0003] In order to overcome the defects in the above-mentioned prior art, the present invention provides a method for visual inspection of pipe straightness, which can realize online inspection of the pipe moving forward along the axial direction by using two cameras. The method is fast, high in precision, simple in device, economical and effective, and can greatly improve production efficiency.

[0004] To achieve the above object, the present invention adopts the following technical solutions, including:

[0005] A method for visually inspecting the straightness of a pipe, which is used to inspect the straightness of the pipe during its forward axial movement. Two cameras with mutually perpendicular imaging planes are arranged at the circumference of the pipe, the main optical axes of the two cameras are perpendicular to the axial direction of the pipe, and the fields of view of the two cameras can completely cover the diameter width of the pipe.

[0006] The detection method is specifically as follows:

[0007] S1. Obtain the intrinsic and extrinsic parameters of the two cameras through calibration and establish a three-dimensional coordinate system;

[0008] S2. Determine the camera's image capturing frequency according to the length and movement speed of the pipe, so that there is an overlap area between adjacent frames of the camera;

[0009] S3, two cameras simultaneously take two images of the pipe according to the imaging frequency, perform feature extraction on the two images of the pipe respectively to obtain the pixel coordinates of the central axis of the pipe, and calculate the three-dimensional coordinate point set of the central axis of the current section of the pipe according to the three-dimensional coordinate system;

[0010] S4, calculating the splicing matrix between adjacent frames according to the overlapped area between adjacent frames, splicing the three-dimensional coordinate point sets of the central axis of multiple sections of the pipe, and obtaining the three-dimensional coordinate point set of the central axis of the entire pipe;

[0011] S5. Perform straight line fitting on the three-dimensional coordinate point set of the center axis of the entire pipe, and use the straight line fitting error to evaluate the straightness of the pipe.

[0012] Preferably, in step S1, one camera is a main camera and the other camera is a secondary camera, and the camera coordinate system of the main camera is assumed to be a three-dimensional coordinate system, that is, the optical center of the main camera is the origin O of the three-dimensional coordinate system, the principal optical axis of the main camera is the Z axis of the three-dimensional coordinate system, and the X axis and Y axis of the three-dimensional coordinate system are parallel to the u axis and v axis of the pixel coordinate system of the main camera; the imaging plane of the secondary camera is parallel to the YOZ plane of the three-dimensional coordinate system.

[0013] Preferably, the telecentric model calibration method is used and a checkerboard calibration plate is used to calibrate the internal parameters of the primary camera and the secondary camera to be M and a 、M b , the pixel coordinate system of the main camera (u a ,v a ) and its camera coordinate system X a ,Y a ,Z a) The mapping relationship between them, and the pixel coordinate system of the secondary camera (u b ,v b ) and its camera coordinate system X b ,Y b ,Z b) The mapping relationships between them are:

[0014]

[0015] By placing a standard sphere between the two cameras, the transformation matrix between the primary camera and the secondary camera, i.e., the external parameter, is obtained, thereby converting the pixel coordinate system (u b ,v b ) is transformed into the camera coordinate system of the main camera, that is, the three-dimensional coordinate system:

[0016] [X a ,Y a ,Z a] T =KX a M b -1 [u b ,v b ,1] T .

[0018] Preferably, in step S2, the field of view of the two cameras along the axial direction of the tube is f, the movement speed of the tube along the axial direction is p, the length of the overlapping area between adjacent frames along the axial direction of the tube is d, and according to the formula Δt=(fd) / v, the time interval for imaging is determined to be Δt, thereby determining the imaging frequency of the camera.

[0019] Preferably, in step S3, two cameras respectively capture an image of the pipe under their respective viewing angles, and edge search is used to obtain pixel point sets of both side edges of the pipe for the captured images, and corresponding points of the side edge pixel points are searched in the opposite side edge pixel point set according to the normal of one side edge to form pixel point pairs of both side edges of the pipe, and the pixel center between the pixel point pairs is calculated as the center of the pipe, thereby obtaining the pixel point set of the central axis of the pipe;

[0020] According to the two pipe images taken by the two cameras, the corresponding pixel point sets of the central axis of the pipe are obtained respectively; then, according to the internal and external parameters of the two cameras and the established three-dimensional coordinate system, the three-dimensional coordinate point set of the central axis of the pipe is calculated.

[0021] Preferably, both cameras use backlit parallel light sources to capture images of the pipe.

[0022] Preferably, in step S4, the three-dimensional coordinate point set of the overlapping area between adjacent frames is obtained, and the ICP alignment method is used to obtain the stitching matrix between adjacent frames; the three-dimensional coordinate point set of the central axis of each section of the pipe is obtained in turn, and the three-dimensional coordinate point sets of the central axis of multiple sections of the pipe are stitched in turn according to the stitching matrix to obtain the three-dimensional coordinate point set of the central axis of the entire pipe.

[0023] Preferably, in step S5, a straight line fitting is performed on the three-dimensional coordinate point set of the central axis of the entire pipe to obtain a straight line equation; the distance from each point in the three-dimensional coordinate point set of the central axis to the straight line equation is calculated to obtain a distance set; the mean of the distance set is used to evaluate the straightness of the pipe, and the smaller the mean, the better the straightness of the pipe.

[0024] The present invention also provides a visual inspection device for straightness of a pipe, which is applicable to the above-mentioned visual inspection method for straightness of a pipe, and the device comprises: two cameras whose imaging planes are perpendicular to each other, two backlight parallel light sources corresponding to the two cameras and perpendicular to each other, a motion module for driving the pipe to move forward along the axial direction, and a computing module for processing images taken by the two cameras;

[0025] The intersection of the fields of view of the two cameras is used to place the pipe, and the axial direction of the pipe is perpendicular to the main optical axes of the two cameras respectively. The fields of view of the two cameras can completely cover the diameter width of the pipe and are used to capture the side image of the pipe;

[0026] The two backlight parallel light sources are used to provide shooting light sources to the two cameras respectively;

[0027] The motion module includes a motor, a rotating shaft and a guide wheel arranged on the rotating shaft; the pipe is placed on the guide wheel, and the motor drives the rotating shaft to rotate, driving the pipe to move forward along the axial direction;

[0028] The calculation module is used to calculate the three-dimensional coordinate point set of the central axis of the pipe according to the images taken by the two cameras, and to evaluate the straightness of the pipe.

[0029] The present invention also provides a computer program product, characterized in that it includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the above-mentioned visual inspection method for straightness of a pipe is implemented.

[0030] The advantages of the present invention are:

[0031] (1) The present invention provides a method for visually inspecting the straightness of a pipe, which can realize online inspection of the pipe in axial forward motion by using two cameras. The method is fast, high in precision, simple in device, economical and effective, and can greatly improve production efficiency.

[0032] (2) The pipe straightness measurement method provided by the present invention is applicable to the field of pipe inspection of various materials. It uses an economical visual method to complete dynamic online inspection of pipes and has the characteristics of high efficiency and high precision.

[0033] (3) Compared with the segmented detection method, this method uses the overlapping area registration and splicing method to calculate the straightness of the entire pipe.

[0034] (4) The pipe straightness detection device provided by the present invention is simple in design, economical and effective, and can be applied to various pipe detection occasions, thereby greatly improving the quality of pipes. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the process of the visual inspection method for pipe straightness of the present invention.

[0036] Figure 2 It is a schematic diagram of a visual inspection device for pipe straightness of the present invention.

[0037] Figure 3 It is a schematic diagram of an imaging device for detecting straightness of a steel pipe in a specific embodiment of the present invention.

[0038] Description of reference numerals:

[0039] 1a and 1b are two mutually perpendicular backlight parallel light sources, 2a and 2b are two mutually perpendicular cameras with imaging planes, 3 is a pipe, 4 is a rotating shaft, and 5 is a computing module. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example 1

[0042] like Figure 2-3 As shown, a visual inspection device for straightness of a pipe comprises: two cameras 2a, 2b whose imaging planes are perpendicular to each other, two backlight parallel light sources 1a, 1b corresponding to the two cameras 2a, 2b respectively and perpendicular to each other, a motion module for driving a pipe 3 to move forward axially, and a computing module 5 for processing images taken by the two cameras.

[0043] The intersection of the fields of view of the two cameras 2a and 2b is used to place the pipe 3 to be tested. The axial direction of the pipe 3 is perpendicular to the main optical axes of the two cameras 2a and 2b respectively. The fields of view of the two cameras 2a and 2b can completely cover the diameter width of the pipe 3 and are used to capture the side image of the pipe 3.

[0044] The two backlight parallel light sources 1a and 1b are used to provide shooting light sources to the two cameras 2a and 2b respectively. The two backlight parallel light sources 1a and 1b and the two cameras 2a and 2b constitute a visual module.

[0045] The motion module includes a motor, a rotating shaft 4 and a guide wheel arranged on the rotating shaft 4; the pipe 3 is placed on the guide wheel of the rotating shaft 4, and the pipe 3 is supported by multiple rotating shafts 4; the motor drives the rotating shaft 4 to rotate, driving the pipe 3 to move forward along the axial direction.

[0046] The calculation module 5 is used to calculate the three-dimensional coordinate point set of the central axis of the pipe according to the images taken by the two cameras 2a and 2b, and to evaluate the straightness of the pipe.

[0047] Example 2

[0048] Based on the detection device of Example 1, Figure 1 As shown, a method for visually inspecting the straightness of a pipe comprises the following steps:

[0049] S1, obtain the intrinsic and extrinsic parameters of the two cameras through calibration and establish a three-dimensional coordinate system.

[0050] Among them, camera 2a is recorded as the main camera, and camera 2b is recorded as the secondary camera. Assume that the camera coordinate system of the main camera is the main coordinate system, that is, the three-dimensional coordinate system, that is, the optical center of the main camera is the origin O of the three-dimensional coordinate system, the main optical axis of the main camera is the Z axis of the three-dimensional coordinate system, and the X axis and Y axis of the three-dimensional coordinate system are parallel to the u axis and v axis of the pixel coordinate system of the main camera; the imaging plane of the secondary camera is parallel to the YOZ plane of the three-dimensional coordinate system.

[0051] Through the telecentric model calibration method and the GP290 chessboard calibration plate, the internal parameters of the main camera and the secondary camera are calibrated as M a 、M b , the pixel coordinate system of the main camera (u a ,v a ) and its camera coordinate system (X a ,Y a ,Z a The mapping relationship between them, and the pixel coordinate system of the secondary camera (u b ,v b ) and its camera coordinate system (X b ,Y b ,Z b ) are as follows:

[0052] [ X a ,Y a ] T =M a -1 [u a ,v a ,1] T

[0053] [ X b ,Y b ] T =M b -1 [u b ,v b ,1] T

[0054] Since the imaging planes of the two cameras are perpendicular to each other, the same point in the camera coordinate systems of the two cameras satisfies the following coordinate relationship:

[0055] Z b =X a

[0056] Z a =X b

[0057] The camera coordinate system where camera 2a is located is used as the main coordinate system. A standard sphere is placed between the two cameras 2a and 2b. Photos are taken at 9 different positions of the standard sphere. The two cameras obtain the pixel coordinates of the sphere respectively. Since the center of the sphere is unique, the conversion matrix K between the main camera and the secondary camera is calibrated, so that the pixel coordinate system of the secondary camera (u b ,v b ) is transformed into the camera coordinate system of the main camera, that is, the three-dimensional coordinate system:

[0058] [X a ,Y a ,Z a] T =KX a M b -1 [u b ,v b ,1] T

[0059] S2. Determine the camera's image-taking frequency according to the length and movement speed of the pipe, so that there is an overlapped area between adjacent frames of the camera.

[0060] Among them, the field of view of the two cameras along the axial direction of the tube is f, the movement speed of the tube along the axial direction is p, and the length of the overlapping area between adjacent frames along the axial direction of the tube is d. According to the formula Δt=(fd) / v, the time interval of imaging is determined to be Δt, thereby determining the imaging frequency of the camera.

[0061] S3. Two cameras simultaneously take two images of the pipe according to the imaging frequency, perform feature extraction on the two pipe images respectively to obtain the pixel coordinates of the central axis of the pipe, and calculate the three-dimensional coordinate point set of the central axis of the current section of the pipe according to the three-dimensional coordinate system.

[0062] The backlight parallel light source used provides a high-contrast lighting effect on the edge of the pipe, so that the two cameras respectively take a pipe image from their respective perspectives, and the edge search method is used to obtain the pixel point sets on the edges of both sides of the pipe. According to the normal of one edge, the corresponding points of the edge pixels on the opposite edge are found in the pixel point set of the opposite edge to form pixel point pairs on the edges of both sides of the pipe, and the pixel center between the pixel point pairs is calculated as the center of the pipe, thereby obtaining the pixel point set of the central axis of the pipe.

[0063] The pixel point set of the central axis of the pipe is obtained simultaneously by the primary camera and the secondary camera, and the three-dimensional coordinate point set of the central axis of the pipe is calculated according to the formula of step S1.

[0064] S4. Calculate the splicing matrix between adjacent frames according to the overlapping area between adjacent frames, splice the three-dimensional coordinate point sets of the central axis of multiple sections of the pipe, and obtain the three-dimensional coordinate point set of the central axis of the entire pipe.

[0065] Since there is a fixed overlap area between adjacent frames, the three-dimensional coordinate point sets calculated in accordance with step S3 for the overlap areas on the adjacent frame images should be completely overlapped. The splicing matrix between adjacent frames is obtained by applying the ICP registration method to the three-dimensional coordinate point sets of the overlap areas between adjacent frames; the three-dimensional coordinate point sets of the central axis of each section of the pipe are obtained in sequence, and spliced ​​in sequence according to the splicing matrix to obtain the three-dimensional coordinate point set of the central axis of the entire pipe.

[0066] S5. Perform straight line fitting on the three-dimensional coordinate point set of the center axis of the entire pipe, and use the straight line fitting error to evaluate the straightness of the pipe.

[0067] Perform straight line fitting on the three-dimensional coordinate point set of the center axis of the entire pipe to obtain the straight line equation; calculate the distance from each point in the three-dimensional coordinate point set of the center axis to the straight line equation to obtain a distance set; use the mean of the distance set to evaluate the straightness of the pipe, the smaller the mean, the better the straightness of the pipe.

[0068] The calculation module 5 is responsible for process control and calculation. Specifically, it is used to control the camera to collect images, calculate calibration parameters, calculate the pixel point set of the central axis of the pipe, and calculate the three-dimensional coordinate point set of the central axis of the pipe. Figure 1 As shown in the method flow, in this embodiment, image processing is developed using the OpenCV image processing library.

[0069] Example 3

[0070] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the decision-making behavior decision method according to various embodiments of the present application described in the above-mentioned embodiment 2 of this specification.

[0071] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0072] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for visually inspecting the straightness of a pipe, characterized in that: Used to detect the straightness of the pipe during its axial forward movement, two cameras with mutually perpendicular imaging planes are arranged at the circumference of the pipe, the main optical axes of the two cameras are perpendicular to the axial direction of the pipe, and the fields of view of the two cameras can completely cover the diameter width of the pipe; The detection method is specifically as follows: S1. Obtain the intrinsic and extrinsic parameters of the two cameras through calibration and establish a three-dimensional coordinate system; S2. Determine the camera's image capturing frequency according to the length and movement speed of the pipe, so that there is an overlap area between adjacent frames of the camera; S3, two cameras simultaneously take two images of the pipe according to the imaging frequency, perform feature extraction on the two images of the pipe respectively to obtain the pixel coordinates of the central axis of the pipe, and calculate the three-dimensional coordinate point set of the central axis of the current section of the pipe according to the three-dimensional coordinate system; S4, calculating the splicing matrix between adjacent frames according to the overlapped area between adjacent frames, splicing the three-dimensional coordinate point sets of the central axis of multiple sections of the pipe, and obtaining the three-dimensional coordinate point set of the central axis of the entire pipe; S5. Perform straight line fitting on the three-dimensional coordinate point set of the center axis of the entire pipe, and use the straight line fitting error to evaluate the straightness of the pipe.

2. A method for visually inspecting straightness of a pipe according to claim 1, characterized in that: In step S1, one of the cameras is a main camera and the other is a secondary camera. The camera coordinate system of the main camera is assumed to be a three-dimensional coordinate system, that is, the optical center of the main camera is the origin O of the three-dimensional coordinate system, the main optical axis of the main camera is the Z axis of the three-dimensional coordinate system, and the X axis and Y axis of the three-dimensional coordinate system are parallel to the u axis and v axis of the pixel coordinate system of the main camera; the imaging plane of the secondary camera is parallel to the YOZ plane of the three-dimensional coordinate system.

3. A method for visually inspecting straightness of a pipe according to claim 2, characterized in that: Through the telecentric model calibration method and the chessboard calibration plate, the internal parameters of the main camera and the secondary camera are obtained as M a 、M b , the pixel coordinate system of the main camera (u a ,v a ) and its camera coordinate system ( X a ,Y a ,Z a) The mapping relationship between them, and the pixel coordinate system of the secondary camera (u b ,v b ) and its camera coordinate system ( X b ,Y b ,Z b) The mapping relationships between them are: By placing a standard sphere between the two cameras, the transformation matrix between the primary camera and the secondary camera, i.e., the external parameter, is obtained, thereby converting the pixel coordinate system (u b ,v b ) is transformed into the camera coordinate system of the main camera, that is, the three-dimensional coordinate system: [X a ,Y a ,Z a] T =KX a M b -1 [you] b ,v b ,1] T 。 4. A method for visually inspecting straightness of a pipe according to claim 1, characterized in that: In step S2, the field of view of the two cameras along the axial direction of the tube is f, the movement speed of the tube along the axial direction is p, and the length of the overlapping area between adjacent frames along the axial direction of the tube is d. According to the formula Δt=(fd) / v, the time interval of imaging is determined to be Δt, thereby determining the imaging frequency of the camera.

5. A method for visually inspecting straightness of a pipe according to any one of claims 1 to 3, characterized in that: In step S3, two cameras respectively take a tube image from their respective viewing angles, and use edge search to obtain pixel point sets on both sides of the tube edge. According to the normal of one side edge, corresponding points of the side edge pixel points are found in the opposite side edge pixel point set to form pixel point pairs on both sides of the tube edge. The pixel center between the pixel point pairs is calculated as the tube center, thereby obtaining the pixel point set of the central axis of the tube. According to the two pipe images taken by the two cameras, the corresponding pixel point sets of the central axis of the pipe are obtained respectively; then, according to the internal and external parameters of the two cameras and the established three-dimensional coordinate system, the three-dimensional coordinate point set of the central axis of the pipe is calculated.

6. A method for visually inspecting straightness of a pipe according to claim 5, characterized in that: Both cameras use backlit parallel light sources to capture images of the tube.

7. A method for visually inspecting straightness of a pipe according to claim 1, characterized in that: In step S4, the three-dimensional coordinate point set of the overlapping area between adjacent frames is obtained, and the ICP registration method is used to obtain the stitching matrix between adjacent frames; the three-dimensional coordinate point set of the central axis of each section of the pipe is obtained in turn, and the three-dimensional coordinate point sets of the central axis of multiple sections of the pipe are stitched in turn according to the stitching matrix to obtain the three-dimensional coordinate point set of the central axis of the entire pipe.

8. A method for visually inspecting straightness of a pipe according to claim 1, characterized in that: In step S5, a straight line fitting is performed on the three-dimensional coordinate point set of the central axis of the entire pipe to obtain a straight line equation; the distance from each point in the three-dimensional coordinate point set of the central axis to the straight line equation is calculated to obtain a distance set; the mean of the distance set is used to evaluate the straightness of the pipe, and the smaller the mean, the better the straightness of the pipe.

9. A visual inspection device for straightness of a pipe, characterized in that: A method for visually inspecting the straightness of a pipe according to any one of claims 1 to 8, the device comprising: two cameras whose imaging planes are perpendicular to each other, two backlight parallel light sources corresponding to the two cameras and perpendicular to each other, a motion module for driving the pipe to move forward along the axial direction, and a computing module for processing images captured by the two cameras; The intersection of the fields of view of the two cameras is used to place the pipe, and the axial direction of the pipe is perpendicular to the main optical axes of the two cameras respectively. The fields of view of the two cameras can completely cover the diameter width of the pipe and are used to capture the side image of the pipe; The two backlight parallel light sources are used to provide shooting light sources to the two cameras respectively; The motion module includes a motor, a rotating shaft and a guide wheel arranged on the rotating shaft; the pipe is placed on the guide wheel, and the motor drives the rotating shaft to rotate, driving the pipe to move forward along the axial direction; The calculation module is used to calculate the three-dimensional coordinate point set of the central axis of the pipe according to the images taken by the two cameras, and to evaluate the straightness of the pipe.

10. A computer program product, characterized in that It includes a computer program / instruction, which, when executed by a processor, implements a method for visually inspecting the straightness of a pipe as described in any one of claims 1 to 8.

Images