Visual measurement system and path planning method for cleaning corner edge defects of continuous casting billets

Abstract

The application provides a kind of visual measurement system and path planning method for cleaning corner edge defects of continuous casting billet, comprising: a plurality of equidistantly arranged fixed cameras along the length direction of continuous casting billet, a plurality of laser emitters corresponding to the fixed cameras, a track arranged along the workpiece arrangement direction, a mobile camera, a network switch, a guide rail and an oxygen nozzle moving on the guide rail, the laser emitters project arrayed line structured light to the surface of continuous casting billet, the shooting range of each fixed camera covers the edge in the width direction of continuous casting billet on the surface of continuous casting billet, and the shooting ranges of adjacent two fixed cameras do not have a shooting overlap area; the fixed camera, the mobile camera and the laser emitter are connected with the network switch, and the network switch is connected with the controller signal; the application can ensure the positioning accuracy of corner point and edge in long-term use, realize the automatic three-dimensional coordinate positioning of corner edge of continuous casting billet, reduce the processing error and the cost required for multiple maintenance and repair.

Description

Technical Field

[0001] This invention relates to the field of billet finishing, specifically to a visual measurement system and path planning method for cleaning edge defects at the corners of continuously cast billets. Background Technology

[0002] Automatic cleaning of continuously cast billets has always been a challenge for steel mills. Currently, large-scale cleaning is generally done manually or mechanically. First, manual cleaning of edge defects is a highly technical task. Operators must undergo professional training, and certain positions require special operation certificates. Furthermore, the labor intensity of cleaning the edges and chamfers of continuously cast billets is very high, resulting in low overall efficiency.

[0003] Secondly, although machine chamfering and cleaning can effectively replace manual labor, large-scale automatic cleaning may lead to waste of the blank material, as unnecessary parts are cut, and large-scale cleaning may increase production costs.

[0004] In addition, the existing mechanical cleaning method uses mechanical scanning positioning, which has insufficient scanning accuracy and accumulates a large motion error during the back-and-forth movement of the nozzle. Over time, due to external vibration, equipment aging and other changes, the mapping accuracy of this mechanical scanning system gradually decreases, resulting in inconsistent overall cleaning quality. Furthermore, manual inspection and secondary manual cleaning are required to meet the quality requirements of the next process. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the technical problem to be solved by the present invention is to provide a visual measurement system and path planning method for cleaning corner edge defects of continuously cast billets, which can ensure the maintenance of corner and edge positioning accuracy during long-term use, realize automated three-dimensional coordinate positioning of corner edges of continuously cast billets, and reduce processing errors and the cost of multiple maintenance and rework.

[0006] To achieve the above objectives, the present invention provides a visual measurement system for cleaning corner edge defects of continuously cast billets, used for visual measurement of corner edges of continuously cast billets, comprising: multiple fixed cameras equidistantly arranged along the length direction of the continuously cast billet, multiple laser emitters corresponding one-to-one with the fixed cameras, a track arranged along the workpiece arrangement direction, a mobile camera moving on the track, a network switch, a guide rail, and a nozzle moving on the guide rail. The laser emitters project an array of line structured light onto the surface of the continuously cast billet. The shooting range of each fixed camera on the surface of the continuously cast billet covers the edge of the continuously cast billet in the width direction, and the shooting ranges of two adjacent fixed cameras do not have overlapping shooting areas. The fixed cameras, the mobile cameras, and the laser emitters are all connected to the network switch, and the network switch is connected to a controller signal.

[0007] A path planning method for cleaning edge defects at the corners of continuously cast billets: This method employs a vision measurement system as described above and includes the following steps:

[0008] S1. A first calibration plate is provided with a rectangular array of multiple first-type concentric circular targets. Single-camera calibration is performed based on the first calibration plate to determine the external parameter matrix [R|t], internal parameter matrix M, and distortion coefficients of each fixed camera.

[0009] S2. The laser emitter projects multiple line structured beams, and in conjunction with the first calibration plate, determines the light plane equations of all line structured beams.

[0010] S3. The moving camera moves multiple times, ensuring that there is a moving camera between every two fixed cameras; using binocular positioning with the moving camera between two adjacent fixed cameras, the extrinsic parameter matrix [R|t] between each pair of adjacent fixed cameras is obtained from the extrinsic parameter matrix [R|t]. lr |t lr ];

[0011] By setting a fixed camera as the reference camera, the extrinsic parameter matrix [R] of the remaining fixed cameras to the reference camera is finally obtained. n1 |t n1 ];

[0012] S4. Obtain the extrinsic parameter matrix [R] from the reference camera coordinate system to the base coordinate system. X |t X ], and by [R X |t X Multiply by the extrinsic parameter matrix of each fixed camera to the reference camera [R] n1 |t n1 ], and obtain the transformation matrix from each fixed camera to the base coordinate;

[0013] S5. Nozzle coordinate calibration, including:

[0014] S51. Each of the fixed cameras takes a picture of the continuously cast billet;

[0015] S52. Calculate the three-dimensional coordinates of the edge points and corner points of the continuous casting billet within the coordinates of each fixed camera, based on the internal parameter matrix M of a single fixed camera and the light plane equation.

[0016] S53, according to the remaining external parameter matrix [R] of the fixed camera to the reference camera. n1 |t n1 The coordinates of the edge points and corner points of the continuously cast billet in the coordinate system of each camera are transformed to the coordinate system of the reference camera.

[0017] S54. Based on the transformation matrix [R] from the reference camera coordinate system to the base coordinate system... X |t X The coordinates of the edge points and corner points of the continuously cast billet in the reference camera coordinate system are transformed to the base coordinate system.

[0018] S55. Based on the position information of the edge points and corner points of the continuously cast billet in the base coordinate system, control the movement of the nozzle to complete the flame cleaning.

[0019] Preferably, step S1 includes: obtaining the known center C(x,y) of the first type of concentric circular target, and the outer radius r. o and inner circle radius r i The center C of the external ellipse of the pixel coordinate system is obtained from the fixed camera under test. o (x,y) and the center of the inner ellipse C i (x,y), and by Formula I and Formula II By combining the equations, we can obtain the true coordinates of the center of the first type of concentric circular target in the pixel coordinate system. (Formula Ⅲ) All fixed cameras respectively capture images of the first calibration plate in different poses, and the actual coordinate set p of the first type of concentric circular targets in the pixel coordinate system is obtained by formula III. ui (u i ,v i ,1), i=1,2,3Kn; In addition, the coordinate set P of the corresponding first type of concentric circle target in world coordinates is obtained from the preset data. wi (X wi ,Y wi Z wi ,1),i=1,2,3Kn;

[0020] p ui (u i ,v i ,1), P wi (X wi ,Y wi Z wi 1) Substitute the coordinate set into the known camera imaging model sp ui =M[R|t]P wi Thus, the external parameter matrix [R|t] of each fixed camera and the internal parameter matrix M of each fixed camera are obtained.

[0021] Thus, the intrinsic parameter matrices of each of the fixed cameras are obtained.

[0022] Where (u0, v0) are the coordinates of the principal point of the image, f x and f yThese are the scale factors of the u-axis and v-axis in the image pixel coordinate system, respectively, and γ is the non-perpendicularity factor of the u-axis and v-axis.

[0023] Preferably, the distortion coefficients are expressed by the following system of equations IV.

[0024]

[0025] Where k1, k2, and k3 are the radial distortion coefficients of the lens, and p1 and p2 are the tangential distortion coefficients of the lens.

[0026] Preferably, step S2 includes:

[0027] S21, Projecting multi-channel structured light P1 to P N The first calibration plate is provided;

[0028] S22. Move the first calibration plate multiple times within the same plane without moving the light beam. Each time the first calibration plate is moved, the lines connecting the centers of a certain column of concentric circular targets of the first type intersect, L1 and P1. Based on the principle of invariant cross-ratio projection, the intersection point D1 of L1 and P1 is obtained. After moving the first calibration plate multiple times, the intersection points D1 to D2 of the same line structured light P1 with the lines connecting the centers of the first type of concentric circular targets at different positions are obtained. N ;

[0029] S23, passing through different intersection points D N The optical plane equation of the line structured light P1 was obtained by fitting the image.

[0030] S24. Obtain all line structured lights P1 to P using the methods described in steps S21 to S23. N The equation of the light plane.

[0031] Preferably, S22 specifically includes the following:

[0032] Line L1 connects to the center A(X) of a certain type of concentric circular target. aw ,Y aw Z aw ), B(X bw ,Y bw Z bw ), C(X) cw ,Y cw Z cw The line connecting the two points is shown; p1 is the projection of ray P1 onto the image coordinate system, and l1 is the projection of L1 onto the image coordinate system. a(x) a ,y a ), b(x b ,y b ), c(x) c ,y c Let A, B, and C be the projection points of A, B, and C in the image coordinate system;

[0033] D1(X dw ,Y dw Z dw ) is the intersection of line L1 and ray P1; d(x) d ,y d () is the projection point of D1 in the image coordinate system;

[0034] According to the cross-ratio projective invariance, we can obtain

[0035]

[0036]

[0037] Therefore, D1(X) can be obtained. dw ,Y dw Z dw ).

[0038] Preferably, step S3 includes:

[0039] S31. When the moving camera moves to the two fixed cameras; first, calculate the external parameter matrix [R] between the moving camera and one of the fixed cameras. ml |t ml ]; then calculate the extrinsic parameter matrix [R] between the moving camera and the fixed camera on the other side. rm |t rm Finally, the external parameter matrix between the two fixed cameras is obtained:

[0040] S32. Move the moving camera, and based on S31, sequentially obtain the extrinsic parameter matrices between all adjacent fixed cameras; after transformation, obtain the extrinsic parameter matrices from the remaining fixed cameras to the reference camera [R]. n1 |t n1 ];

[0041] S33. Finally, minimize the system reprojection error to obtain the external parameter matrix [R]. n1 |t n1 ] Optimize.

[0042] Preferably, step S31 includes:

[0043] The external parameter matrix of the fixed camera on the left [R] l |t l The mobile camera is [R] r |t r ];

[0044] The coordinates of the center point of the first type of concentric circular target in the world coordinate system are P. wiThe coordinates of P in the fixed camera coordinate system on the left are represented as P. li The coordinates in the moving camera coordinate system are represented as P. mi , expressed as a formula

[0045]

[0046] After transformation, we can obtain P mi =[R m (R l ) -1 ]P li +[t mi -R m (R l ) -1 t li ]

[0047] That is, the external parameter matrix from the fixed camera on the left to the moving camera is [R ml |t ml ];

[0048]

[0049] Similarly, the external parameter matrix from the fixed camera to the moving camera on the right can be obtained as [R rm |t rm ];

[0050] Let P r Let P be a point in the fixed camera coordinate system on the right. Its coordinates in the moving camera coordinate system can be represented as P. m Then its coordinates P in the fixed camera coordinate system on the left are... l The following formula can be used

[0051] Solve

[0052] Parallel P mi =(R ml R rm )P r +(R ml t rm +t ml )

[0053] The external parameter matrix from the right fixed camera coordinate system to the left fixed camera coordinate system can be obtained.

[0054]

[0055] Preferably, S4 includes: establishing a base coordinate system O with the zero position of the guide rail. b -X b Y b Z b Coordinate system;

[0056] Assuming a mounting plate is installed on the nozzle, and the mounting plate is coated with a second type of concentric circular target, with an initial position of i, the nozzle is moved to position i+1; at position i, the transformation relationship of the center point of the second type of concentric circular target from the reference camera coordinate system to the end effector coordinate system is given by formula V.

[0057]

[0058] This indicates the transformation relationship from the end-effector coordinate system to the base coordinate system;

[0059] This indicates the transformation relationship from the second calibration plate coordinate system to the end execution coordinate system;

[0060] This indicates the transformation relationship from the second calibration plate coordinate system to the reference camera coordinate system;

[0061] This indicates the transformation relationship from the reference camera coordinate system to the base coordinate system;

[0062] Since the relative relationship between the second calibration plate coordinate system and the end effector coordinate system remains constant, formula VI can be obtained. From formulas V and VI, we can obtain:

[0063] Right now

[0064] Since the relative position between the reference camera coordinate system and the base coordinate system remains fixed, let's assume...

[0065]

[0066]

[0067]

[0068] Finally obtained

[0069] AX = XB

[0070] A can be obtained directly from the motion parameters of the nozzle relative to the base coordinate system, B can be obtained from the pose of the second calibration plate plane positioned by the camera 1 coordinate system, and X is the external parameter matrix from the reference camera coordinate system to the base coordinate system to be determined.

[0071] As described above, the visual measurement system and path planning method of the present invention have the following beneficial effects:

[0072] This invention enables real-time adjustment of the internal parameter matrix M and distortion coefficients of a fixed camera through multi-camera single-target calibration, avoiding coordinate measurement errors caused by changes in camera parameters over long-term use, thus reducing maintenance and testing costs. Furthermore, the laser emitter in this invention can obtain the required accurate pixel coordinates by determining the light plane equation and combining it with the internal parameter matrix. Additionally, this invention uses a multi-target calibration method with multiple fixed cameras to obtain the transformation matrix between each fixed camera, allowing the measured three-dimensional coordinates to be transformed between the various camera coordinate systems. Finally, the coordinates in the base coordinate system are obtained through the transformation matrix from each fixed camera to the base coordinate system, thereby controlling the nozzle to perform accurate cutting. Attached Figure Description

[0073] Figure 1 This is a schematic diagram of the structure of the vision measurement system of the present invention;

[0074] Figure 2 A schematic diagram of the first calibration plate and the first type of concentric circle target pattern on it;

[0075] Figure 3 A schematic diagram of the second calibration plate and the second type of concentric circle target pattern on it;

[0076] Figure 4 This refers to the process of centrifugal deviation in the image of concentric circles caused by the perspective projection of the camera.

[0077] Figure 5 This is a schematic diagram showing that the projection cross ratio remains unchanged in S2 of the path planning method for cleaning edge defects at the corner of a continuously cast billet according to the present invention.

[0078] Figure 6 This is a schematic diagram showing the relationship between all the fixed cameras and the moving cameras in the visual measurement system of the present invention.

[0079] Component designation explanation

[0080] 01 Continuous casting billet

[0081] 02 Fixed Camera

[0082] 1 camera 1

[0083] 2 cameras 2

[0084] 3 cameras

[0085] 4 cameras

[0086] 5 cameras

[0087] 6 cameras

[0088] 7 cameras 7

[0089] 8. Moving camera

[0090] 03 Laser emitter

[0091] 04 Track

[0092] 05 Network Switch

[0093] 06 Guide Rail

[0094] 07 Nozzle

[0095] 08 Controller Detailed Implementation

[0096] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification.

[0097] Please see Figures 1 to 6 It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings of this specification are merely for illustrative purposes to aid those skilled in the art and are not intended to limit the scope of the invention. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to size, without affecting the effectiveness and purpose of the invention, should still fall within the scope of the technical content disclosed in this invention. Furthermore, the terms such as "upper," "lower," "left," "right," "middle," and "one" used in this specification are merely for clarity and are not intended to limit the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.

[0098] like Figure 1 As shown, the present invention provides a visual measurement system for cleaning edge defects at the corners of continuously cast billets, comprising:

[0099] Multiple fixed cameras 02 are equidistantly arranged along the length of the continuous casting billet 01, multiple laser emitters 03 corresponding one-to-one with the fixed cameras 02, a track 04 is set along the workpiece arrangement direction, a mobile camera 8 moves on the track 04, a network switch 05, a guide rail 06, and a nozzle 07 moves on the guide rail 06. The laser emitters 3 project an array of line structured light onto the surface of the continuous casting billet 1. The shooting range of each fixed camera 2 on the surface of the continuous casting billet 1 covers the edge of the continuous casting billet 1 in the width direction, and the shooting ranges of two adjacent fixed cameras 2 do not have overlapping shooting areas. The fixed cameras 2, the mobile cameras 5, and the laser emitters 3 are all connected to the network switch 05, and the network switch 05 is signal-connected to the controller 08.

[0100] This invention uses a fixed camera 2 to photograph the surface of the continuously cast billet 1, thereby capturing the edge points and corner points of the continuously cast billet 1 that need to be cut. However, since the position of the starting corner point of the continuously cast billet under the field of view of the fixed camera 2 is uncertain, it is necessary to determine the edge points and corner points through an algorithm, which is specifically reflected in the path planning method below. In addition, the movable camera 5 set in this invention can help transform the coordinate system of two adjacent fixed cameras 2. Furthermore, the guide rail 06 and nozzle 07 set in this invention enable the nozzle 07 to cut the edge points and corner points of the continuously cast billet 1.

[0101] This invention also provides a path planning method for cleaning edge defects at the corners of continuously cast billets, using the vision measurement system described above, and includes the following steps:

[0102] S1. Based on the first calibration plate with a rectangular array of first-type concentric circular targets, such as... Figure 2 As shown, single-camera calibration is performed to determine the external parameter matrix [R|t], internal parameter matrix M, and distortion coefficients of each fixed camera 02;

[0103] S2. Laser emitter 03 projects multiple line structured beams, and in conjunction with the first calibration plate, determines the light plane equations of all line structured beams.

[0104] S3. The moving camera 8 moves multiple times, ensuring that there is one moving camera 8 between every two fixed cameras 02; using the moving camera 8 in two adjacent fixed cameras 02 for binocular positioning, the external parameter matrix [R|t] between every two adjacent fixed cameras 02 is obtained from the external parameter matrix [R|t] of each of the two adjacent fixed cameras 02. lr |t lr ];

[0105] By setting a fixed camera 02 as the reference camera, the extrinsic parameter matrix [R] of the remaining fixed cameras to the reference camera is finally obtained. n1 |t n1 ];

[0106] S4. Obtain the external parameter matrix [R] from the reference camera coordinate system to the base coordinate system. X |t X ], and by [R X |t X Multiply by the extrinsic parameter matrix of each fixed camera to the reference camera [R] n1 |t n1 ], and obtain the transformation matrix from each fixed camera (02) to the base coordinate;

[0107] S5. Nozzle coordinate calibration, including:

[0108] S51. Each fixed camera 02 takes pictures of the continuously cast billet 01.

[0109] S52. Based on the internal parameter matrix M of a single fixed camera 02 and the light plane equation, calculate the three-dimensional coordinates of the edge points and corner points of the continuous casting billet 01 within the coordinates of each fixed camera 02.

[0110] S53, Based on the external parameter matrix from the remaining fixed cameras to the reference camera [R] n1 |t n1 The coordinates of the edge points and corner points of the continuously cast billet 01 in each camera coordinate system are transformed to the coordinate system of the reference camera.

[0111] S54. Based on the transformation matrix [R] from the reference camera coordinate system to the base coordinate system X |t X The coordinates of the edge points and corner points of the continuously cast billet 01 in the reference camera coordinate system are transformed to the base coordinate system.

[0112] S55. Based on the position information of the edge points and corner points of the continuously cast billet 01 in the base coordinate system, control the movement of the nozzle 07 to complete the cutting.

[0113] In S1 above, the methods for obtaining the extrinsic parameter matrix [R|t], intrinsic parameter matrix M, and distortion coefficients of each fixed camera 02 are as follows:

[0114] like Figure 1 As shown, there are 7 fixed cameras (1 to 7) equidistantly arranged along the length of the continuously cast billet 1, named from left to right as the first fixed camera 1 to the seventh fixed camera 7; for ease of writing, the following writing method is adopted: the first fixed camera is camera 1, the second fixed camera is camera 2, and so on;

[0115] Cameras 1 through 7 are calibrated using the first calibration plate, such as... Figure 2 As shown, taking the center of the first type of concentric circular target as the feature point, since in the perspective imaging of the camera, the inner and outer circles of the first type of concentric circular target appear as two ellipses with separate centers, as shown... Figure 4 As shown, therefore, it is first necessary to correct the eccentricity error of the elliptical pairs imaged by the first type of concentric circular targets. C(x,y) is the center of the first type of concentric circular targets, r o and r i C is the radius of the outer and inner circles; C'(x',y') is the perspective point of C after perspective transformation; C o (x o ,y o ) and C i (x i ,y i These are the centers of the outer and inner ellipses, respectively. The eccentricity errors of the outer and inner ellipses are described as follows:

[0116]

[0117]

[0118] Among them, K x K y The two coefficients are related to the radius r of the first type of concentric circle target. o and r i It is irrelevant, and for the first type of concentric circle target, the inner and outer circles are equal. Combining the two formulas above, the coordinates of the true projection center point of the first type of concentric circle target can be obtained:

[0119]

[0120] Cameras 1 through 7 respectively acquire images of the first calibration board in different poses (the calibration board is placed on the same plane), and perform eccentricity error correction. Then, the coordinates of the true projection center point of the first type of concentric circular target are obtained on the image (in pixel coordinate system at this time).

[0121] p ui (u i ,v i ,1),i=1,2,3Kn (4)

[0122] Since the first calibration plate is custom-made, and the physical position and size of the first type of concentric circular targets are known, a world coordinate system is established on the first calibration plate. Hereafter, the first calibration plate refers to an array with the first type of concentric circular targets. Thus, the world coordinates of the center point of the first type of concentric circular targets can be obtained.

[0123] P wi (X wi ,Y wi Z wi ,1),i=1,2,3Kn (5)

[0124] Furthermore, based on the existing camera imaging model (6), substituting the coordinate set p... ui (u i ,v i ,1) and P wi (X wi ,Y wi Z wi ,1); The internal parameter matrix M and external parameter matrix [R|t] of camera 1 to camera 7 can be obtained. The specific solution process will not be described in detail here.

[0125] The existing technology is titled A, Flexible, New, Technique, for, Camera, Calibration, authored by Zhengyou Zhang; see the following URL for details:

[0126] https: / / www.microsoft.com / enus / research / ?from=http%3A%2F%2Fresearch.microsoft.com%2F%25cb%259czhang,;

[0127] sp ui =M[R|t]P wi (6)

[0128] Where s is the scaling factor, and R and t are the rotation matrix and translation vector of the target from the world coordinate system to the camera coordinate system;

[0129] M is the camera's intrinsic parameter matrix, written as...

[0130]

[0131] Where (u0, v0) are the coordinates of the principal point of the image, f x and f y These are the scale factors of the u-axis and v-axis in the image pixel coordinate system, respectively, and γ is the non-perpendicularity factor of the u-axis and v-axis.

[0132] Considering the distortion inherent in camera lenses, we need to account for radial and tangential distortion. Let point P be an example. wi Ideal imaging point p i The coordinates in the image coordinate system are (x i ,y i (As shown in the image coordinate system before distortion correction), point P wi The actual imaging point p i The coordinates in the image coordinate system are (x d ,y d (Distortion corrected and displayed in the image coordinate system), the image distortion is corrected according to the following distortion model.

[0133]

[0134] Where k1, k2, and k3 are the radial distortion coefficients of the lens, and p1 and p2 are the tangential distortion coefficients of the lens.

[0135] In summary, by processing the first calibration board images in different poses, the intrinsic parameter matrix M and distortion coefficients k1, k2, k3, p1, p2 of a single camera can be obtained.

[0136] The specific derivation of the equation for the light plane in S2 is as follows:

[0137] By calibrating all line structured lights emitted by the laser emitter, the optical plane equations for all line structured lights are obtained.

[0138] Specifically, this invention is based on the principle of cross-ratio projection invariance in camera perspective projection, and uses the plane of the first calibration plate to achieve line structure cursor calibration, such as... Figure 5 As shown, where

[0139]

[0140] In actual operation, the camera captures a planar image of the calibration plate projecting the line structured light, processes it, and detects the center image coordinates of the first type of concentric circular targets based on the method described in step S1. Multiple horizontal straight lines are obtained by connecting the centers row by row. The center line of the line structured light is detected on the image, and the image coordinates of multiple intersection points between the center line and the horizontal straight lines are obtained. Since the world coordinates of the center points of the first type of concentric circular targets are known, the world coordinates of the intersection points can be obtained based on the cross-ratio invariance in the perspective projection. Based on the transformation matrix from the world coordinate system to the camera coordinate system obtained from the single-target calibration, the world coordinates of the intersection points are transformed to the camera coordinate system. After multiple operations, the three-dimensional coordinates of multiple points on the line structured light plane in the camera coordinate system are obtained, and the light plane equation of the line structured light can be fitted. A specific explanation is given below using a line structured light 68 within the field of view of camera 1 as an example:

[0141] Figure 5 As shown, during structured light plane calibration, the image points corresponding to feature points A, B, and C on the target plane are a, b, and c, respectively. The structured light plane intersects the target plane to form a straight line L. 68 And the straight line L 68 It intersects with line E1 at the feature point D, and the corresponding image point is d.

[0142] Let A(X) aw ,Y aw Z aw ), B(X bw ,Y bw Z bw ), C(X) cw ,Y cw Z cw ), D(X dw ,Y dw Z dw Let a(x) be the world coordinates corresponding to feature points A, B, and C. a ,y a ), b(x b ,y b ), c(x) c ,y c ), d(x d ,y d () represents the image coordinates of the feature point. Here, a, b, and c can be obtained by locating the center of the first type of concentric circular target, and the image coordinates of point d can be obtained by fitting the straight line L.68 The intersection point of the line E1 and the line E1 is obtained. Given the world coordinates of feature points A, B, and C, the formula for determining the world coordinates of the point D can be obtained by applying the constant cross ratio:

[0143]

[0144]

[0145] Following this method, multiple similar parallel straight lines can be drawn on the calibration plate plane. By moving the target plane multiple times along the direction of the light plane, a line L can be obtained. 68 Multiple non-collinear feature points intersect to fit the light plane, while ensuring that the positions of the camera and laser remain unchanged.

[0146] Similarly, by performing a similar operation on each line structured light, the light plane equations for line structured lights 68 to 123 can be obtained:

[0147] aX wi +bY wi +cZ wi +d=0 (12)

[0148] Therefore, by combining the camera imaging equation obtained in step S1 and the light plane equation obtained in step S2, the three-dimensional coordinates of any point on the light plane in the corresponding camera coordinate system can be solved.

[0149] The external parameter matrices of the remaining fixed cameras 02 to the reference camera in S3 [R] ml |t ml The process of obtaining the result is as follows;

[0150] Since the camera coordinate systems are independent of each other, the measured coordinates are not uniform. Therefore, it is necessary to solve the transformation relationship between the camera coordinate systems and unify the obtained coordinates to the reference camera coordinate system. In the system described in this invention, the coordinate system of camera 1 is regarded as the reference camera coordinate system. A movable camera 8 is added as an auxiliary camera for multi-target calibration. Since the real-time intelligent cutting path planning task of continuous casting billet has a large field of view, cameras 1 to 7 are arranged linearly to increase the overall field of view of the system. There is no common field of view between adjacent cameras. In order to facilitate multi-target calibration, a movable camera 8 is added and placed between two adjacent cameras in sequence. It serves as an auxiliary tool to indirectly obtain the external parameter matrix between adjacent camera coordinate systems. The external parameter matrix from the coordinate systems of camera 2 to camera 7 to the coordinate system of camera 1 is obtained. Finally, the obtained external parameter matrix is ​​optimized by minimizing the system reprojection error to complete the calibration. The process is illustrated as follows. Figure 6 As shown. The following is a detailed explanation using camera 1 and camera 2 as examples.

[0151] like Figure 6As shown, there is no common field of view between camera 1 and camera 2, so the external parameter matrix from the camera 2 coordinate system to the camera 1 coordinate system cannot be obtained directly through binaural calibration. Therefore, a moving camera 8 is installed between camera 1 and camera 2. At this time, the moving camera 8 has a large common field of view with both camera 1 and camera 2. First, binaural calibration is performed on the moving camera 8 and camera 1 to obtain the external parameter matrix from the moving camera 8 coordinate system to the camera 1 coordinate system [R]. 81 |t 81 Then, dual-target positioning is performed on moving camera 8 and camera 2 to obtain the extrinsic parameter matrix [R] from the coordinate system of camera 2 to the coordinate system of moving camera 8. 28 |t 28 [R] obtained based on dual-objective determination 81 |t 81 ] and [R 28 |t 28 By doing so, the external parameter matrix from camera 2 coordinate system to camera 1 coordinate system can be obtained.

[0152] Taking camera 1 and moving camera 8 as an example, the dual-target positioning method is explained in detail below:

[0153] Using the first type of concentric circular target plane target, based on the single-target calibration described in step S11, the external parameter matrices of the target from the world coordinate system (i.e., the coordinate system established on the first calibration plate as described in step S11) to the camera 1 coordinate system and the moving camera 8 coordinate system can be obtained respectively. l |t l ] and [R r |t r The coordinates of the center point of the first type of concentric circle target in the world coordinate system are P. wi The coordinates in the camera 1 coordinate system are represented as P. li The coordinates in the moving camera coordinate system are represented as P. mi Then it can be expressed as

[0154]

[0155] Transforming formula (13), it can be expressed as:

[0156]

[0157] Subtracting the two equations, we get

[0158] (R l ) -1 ·(P li -t li )-(R r ) -1 ·(P ri -t ri )=0 (15)

[0159] Multiplied by R r

[0160] R r (R l ) -1 ·(P li -t li )-(P ri -t ri )=0 (16)

[0161] P ri =R r (R l ) -1 ·(P li -t li )+t ri (17)

[0162] After transformation, we can obtain

[0163] P ri =[R r (R l ) -1 ]P li +[t ri -R r (R l ) -1 t li (18)

[0164] Therefore, the external parameter matrix from the camera 8 coordinate system to the camera 1 coordinate system can be obtained.

[0165]

[0166] In specific operation, camera 1 and moving camera 8 simultaneously acquire images of the first calibration board in different poses. The center point of the first type of concentric circular target in the common field of view is selected as the feature point. Based on the dual-target calibration method, the external parameter matrix [R] from the coordinate system of moving camera 8 to the coordinate system of camera 1 is obtained. 81 |t 81 Similarly, the external parameter matrix [R] from the camera 2 coordinate system to the moving camera 8 coordinate system can be obtained. 28 |t 28 ].

[0167] Let P2 be a point in the coordinate system of camera 2. Its coordinates in the coordinate system of moving camera 8 can be represented as P8. Then its coordinates P1 in the coordinate system of camera 1 can be solved as follows:

[0168]

[0169] Solving the system of equations simultaneously yields the following results:

[0170] P1=(R 81 P 28 P2+R 81 t 28 +t 81 (twenty one)

[0171] Therefore, the external parameter matrix from camera 2 coordinate system to camera 1 coordinate system can be obtained.

[0172]

[0173] At this point, the extrinsic parameter matrix calibration between the camera 1 coordinate system and the camera 2 coordinate system is complete. Similarly, by moving camera 8, the extrinsic parameter matrices from the camera 2 coordinate system to the camera 7 coordinate system and back to the camera 1 coordinate system are obtained sequentially based on the multi-target calibration method.

[0174] To improve calibration accuracy, when performing multi-target timing, the reprojection error E needs to be adjusted. rms The calculation is performed using the following formula.

[0175]

[0176] Among them, (x u y u () are the actual extracted pixel coordinates. It involves reprojecting feature points with known three-dimensional spatial coordinates back onto the image plane according to the calibrated camera parameters, thus obtaining the pixel coordinates of the reprojected points.

[0177] For the system in this invention, the total RMS can be represented as follows:

[0178] E rms_total =E rms_81 +E rms_28 +E rms_82 +E rms_38 +…+E rms_78 (twenty four)

[0179] Using the LM algorithm to analyze E rms_total To find the minimum value, use E rms_total The minimum value is used to optimize the rotation matrix R and translation vector T obtained from the calibration, and the final calibration result is obtained.

[0180] Based on the camera imaging equation (6) solved in step S1, the light plane equation (12) solved in step S2, and the transformation matrix from the camera 2 coordinate system to the camera 7 coordinate system to the camera 1 coordinate system solved in step S33, an image of the continuously cast billet with projected wired structured light is captured and processed to obtain the corresponding light plane equation parameters (A). l B l C l Dl The coordinates of point p(u, v) in the image are given in the three-dimensional coordinates P in the reference camera coordinate system. c (X c Y c Z c The formula is as follows:

[0181]

[0182] The calibration of the camera 1 coordinate system and the base coordinate system in S4 yields the external parameter matrix from the camera 1 coordinate system to the base coordinate system, i.e., the coordinate system of nozzle 07.

[0183] Specifically, in actual operation, the movement of nozzle 07 is generally controlled with reference to the zero position of the guide rail. Therefore, it is necessary to obtain the external parameter matrix from the camera 1 coordinate system to the base coordinate system, so as to transform the three-dimensional coordinates obtained in the aforementioned steps S1, S2, and S3 to the base coordinate system of guide rail 08. A base coordinate system O is established with the zero position on guide rail 08 as the origin. b -X b Y b Z b A second calibration plate is attached above nozzle 07. The second calibration plate has a second type of concentric circle target, such as... Figure 3 As shown, specifically, the second type of concentric circle target has a black rectangular border. Eight small white circles are arrayed on the black ring of the second type of concentric circle target. The centers of the small circles are evenly distributed on a circle of the same radius. The center of the rectangular border, the center of the array of small circles, and the center of the second type of concentric circle target are all at the same point. Figure 3 As shown. The principles for establishing the coordinate system are as follows: First, the center of the second type of concentric circular target on the second calibration plate is taken as the origin, and two small circles with a 90-degree angle between the line connecting them and the center of the second type of concentric circular target are selected as the positive directions of the X and Y axes to establish the second calibration plate coordinate system; second, the internal corner points of the rectangular black frame are extracted and set on the Z=0 plane, and the center of the rectangle (i.e., the center of the second type of concentric circular target) is taken as the origin of the coordinate system. The X and Y coordinate values ​​are determined according to the specifications and dimensions of the calibration plate image to establish the second calibration plate coordinate system. Therefore, the white small circles of the second type of concentric circular target and the rectangular outer frame are used to assist in establishing the Z=0 plane of the three-dimensional coordinate system. The moving nozzle 07 is set so that the center of the second type of concentric circular target on the second calibration plate passes through the line structured light 68 and the remaining line structured lights in sequence and the image is captured. The image is processed to obtain the external parameter matrix from the camera 1 coordinate system to the base coordinate system.

[0184] Establish the end effector coordinate system O on nozzle 07. t -X t Y t Z tThe end-effector coordinate system and the second calibration plate coordinate system (a coordinate system with the center of the second type of concentric circular target on the second calibration plate as its origin) move simultaneously but their relative positions remain unchanged, while the relative positions of the base coordinate system and the camera 1 coordinate system remain unchanged. The transformation relationship between the coordinate systems can be expressed as follows:

[0185] This indicates the transformation relationship from the end-effector coordinate system to the base coordinate system;

[0186] This indicates the transformation relationship from the second calibration plate coordinate system to the end execution coordinate system;

[0187] This indicates the transformation relationship from the second calibration plate coordinate system to the reference camera coordinate system;

[0188] This indicates the transformation relationship from the reference camera coordinate system to the base coordinate system;

[0189] The transformation relationships are all 4×4 homogeneous transformation matrices, which can be described by the rotation matrix R and the translation vector T, i.e.

[0190] Assuming a second calibration plate is mounted on nozzle 07, and the initial position of the second type of concentric circular target on this second calibration plate is i, the nozzle 07 is moved to position i+1 (note: both positions i and i+1 are assumed to satisfy the condition that the center of the second type of concentric circular target is on the online structured light plane). According to the transformation relationship of each part, at position i, the transformation relationship of the center point of the second type of concentric circular target from the camera 1 coordinate system to the end effector coordinate system is as follows:

[0191]

[0192] in, At position i+1, the transformation relationship of the center point of the second type of concentric circular target from the camera 1 coordinate system to the end effector coordinate system is as follows:

[0193]

[0194] in,

[0195] Since the relative relationship between the second calibration plate coordinate system and the end effector coordinate system remains constant, it can be concluded that...

[0196]

[0197] Then it is easy to obtain

[0198]

[0199] Organized

[0200]

[0201] Since the relative positions of camera 1 coordinate system and base coordinate system are fixed, let's assume...

[0202]

[0203]

[0204]

[0205] Organizing can yield

[0206] AX = XB (34)

[0207] Among them, A can be obtained directly from the motion parameters of nozzle 07 relative to the base coordinate system, B can be obtained from the plane pose of the second type of concentric circular target in the camera 1 coordinate system, and X is the external parameter matrix from the camera 1 coordinate system to the base coordinate system to be determined.

[0208] Thus, based on the single-target calibration, line structure light calibration, multi-target calibration, and calibration from camera 1 coordinate system to base coordinate system, the internal parameter matrix M and distortion coefficients of a single fixed camera 02, the line structure light plane equation, the transformation matrix from multiple fixed camera 02 coordinate systems to the reference camera coordinate system, and the transformation matrix from the reference camera coordinate system to the base coordinate system can be obtained.

[0209] In actual operation, each fixed camera 02 photographs the continuous casting billet 01, and at this time, multi-channel structured light with the previously determined light plane equation is projected onto the continuous casting billet 01.

[0210] The controller 08 processes the captured images to obtain the pixel coordinates of the corner and edge points of the continuous casting billet; then, based on the internal parameter matrix M of a single fixed camera 02, the light plane equation, and the obtained pixel coordinates, the three-dimensional coordinates of the edge and corner points of the continuous casting billet (01) within the coordinates of each fixed camera 02 are calculated.

[0211] According to the multi-target positioning method (i.e., step S3), based on the extrinsic parameter matrix [R] of the other fixed cameras transformed to the reference camera... n1 |t n1 The coordinates of the edge points and corner points of the continuous casting billet 01 in each camera coordinate system are transformed to the coordinate system of the reference camera; according to the transformation matrix from the reference camera coordinate system to the base coordinate system, the coordinates of the edge points and corner points of the continuous casting billet 01 in the reference camera coordinate system are transformed to the base coordinate system; according to the position information of the edge points and corner points of the continuous casting billet 01 in the base coordinate system, the movement of the nozzle 07 is controlled to complete the cutting.

[0212] In summary, this invention, by setting multiple cameras for single-target calibration, can adjust the internal parameter matrix M and distortion coefficients of the fixed camera 02 in real time, avoiding coordinate measurement errors caused by changes in camera parameters due to long-term use, thus reducing maintenance and testing costs. Furthermore, the laser emitter set in this invention can obtain the required accurate pixel coordinates by determining the light plane equation and combining it with the internal parameter matrix. Additionally, this invention obtains the torque matrix between each fixed camera through a multi-target calibration method using multiple fixed cameras 02, enabling the measured three-dimensional coordinates to be transformed between the various camera coordinate systems. Finally, the coordinates in the base coordinate system are obtained through the transformation matrix from each fixed camera 02 to the base coordinate system, thereby controlling the nozzle 07 to perform accurate cutting.

[0213] Therefore, this invention effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0214] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A path planning method for cleaning edge defects at the corners of continuously cast billets, characterized in that: A visual measurement system for cleaning the corner edges of continuously cast billets is used. This system is used to visually measure the corner edges of the continuously cast billet (01). It includes multiple fixed cameras (02) equidistantly arranged along the length of the billet (01), multiple laser emitters (03) corresponding one-to-one with each fixed camera (02), a track (04) arranged along the workpiece arrangement direction, a moving camera (8) moving on the track (04), a network switch (05), a guide rail (06), and a nozzle (07) moving on the guide rail (06). The light emitter (03) projects an array of line structured light onto the surface of the continuous casting billet (01). The shooting range of each fixed camera (02) on the surface of the continuous casting billet (01) covers the edge of the continuous casting billet (01) in the width direction, and the shooting ranges of two adjacent fixed cameras (02) do not have overlapping shooting areas. The fixed cameras (02), the moving camera (8), and the laser emitter (03) are all connected to the network switch (05), and the network switch (05) is signal-connected to the controller (08). The path planning method includes the following steps: S1. A first calibration plate is provided with a rectangular array of multiple first-type concentric circular targets. Single-camera calibration is performed based on the first calibration plate to determine the external parameter matrix of each of the fixed cameras (02). , intrinsic parameter matrix M and distortion coefficients; S2, The laser emitter (03) projects multiple line structured lights, and in conjunction with the first calibration plate, determines the light plane equations of all line structured lights; S3. The moving camera (8) moves multiple times, so that there is a moving camera (8) between every two fixed cameras (02); dual-target positioning is performed by the moving camera (8) in the two adjacent fixed cameras (02), and the external parameter matrices of the two adjacent fixed cameras are used. The external parameter matrix [R] between every two adjacent fixed cameras (02) is obtained. rl |t rl ]; By setting a fixed camera (02) as the reference camera, the external parameter matrix from the other fixed cameras to the reference camera is finally obtained. ; S4. Obtain the extrinsic parameter matrix from the reference camera coordinate system to the base coordinate system. And by Multiply by the extrinsic parameter matrix of each fixed camera to the reference camera. The transformation matrix from each fixed camera (02) to the base coordinate system is obtained; S5. Nozzle coordinate calibration, including: S51. Each of the fixed cameras (02) takes a picture of the continuous casting billet (01); S52. Based on the internal parameter matrix M of a single fixed camera (02) and the light plane equation, calculate the three-dimensional coordinates of the edge points and corner points of the continuous casting billet (01) within the coordinates of each fixed camera (02); S53, Based on the remaining external parameter matrix from the fixed camera to the reference camera. The coordinates of the edge points and corner points of the continuous casting billet (01) in the coordinate system of each camera are transformed to the coordinate system of the reference camera. S54. Based on the external parameter matrix from the reference camera coordinate system to the base coordinate system. The coordinates of the edge points and corner points of the continuous casting billet (01) in the reference camera coordinate system are transformed to the base coordinate system; S55. Based on the position information of the edge points and corner points of the continuously cast billet (01) in the base coordinate system, control the movement of the nozzle (07) to complete the flame cleaning.

2. The path planning method for cleaning edge defects at the corner of a continuously cast billet according to claim 1, characterized in that, Step S1 includes: obtaining the known center of the first type of concentric circular target. and outer circle radius and inner circle radius The center of the outer ellipse of the pixel coordinate system is obtained by the fixed camera (02) under test. and the center of the inner ellipse And by Formula I and Formula II By combining the equations, we can obtain the true coordinates of the center of the first type of concentric circular target in the pixel coordinate system. (Formula Ⅲ) All fixed cameras (02) respectively capture the first calibration plate in different poses, and obtain the actual coordinate set of the first type of concentric circle target in the pixel coordinate system through formula III. Additionally, the set of coordinates of the first type of concentric circle target in world coordinates is obtained from the preset data. ; Will , Substitute the coordinate set into the known camera imaging model Where s is the scaling factor, and R and t are the rotation matrix and translation vector of the target from the world coordinate system to the camera coordinate system; thus, the external parameter matrices of each of the fixed cameras (02) are obtained. and the internal parameter matrix M of each of the fixed cameras (02), ， in The coordinates of the principal point in the image. and These are the scale factors of the u-axis and υ-axis in the image pixel coordinate system, respectively, and γ is the non-perpendicularity factor of the u-axis and υ-axis.

3. The path planning method for cleaning edge defects at the corner of a continuously cast billet according to claim 1, characterized in that, The distortion coefficients are determined by the following set of equations IV: in , and It is the radial distortion coefficient of the lens. and This represents the tangential distortion coefficient of the lens. .

4. The path planning method for cleaning edge defects at the corner of a continuously cast billet according to claim 1, characterized in that, Step S2 includes: S21, Projecting multi-channel structured light to On the first calibration plate; S22. The first calibration plate is moved multiple times within the same plane without moving the light beam; each time the first calibration plate is moved, the center line of a certain column of concentric circular targets of the first type is aligned. and Intersection, based on the principle of cross-ratio projection invariance, is obtained. and intersection After moving the first calibration plate multiple times, the same line of structured light was obtained. Intersection of lines connecting the centers of concentric targets of the first type at different locations to ; S23, passing through different intersection points Fitting to obtain line structured light The equation of the light plane; S24. Obtain all line structured lights using the methods described in steps S21 to S23. to The equation of the light plane.

5. The path planning method for cleaning edge defects at the corner of a continuously cast billet according to claim 4, characterized in that, S22 specifically includes the following: Connection Centered on a certain type I concentric circular target , , The result of connecting the lines; For light Projection in the image coordinate system That is Projection in the image coordinate system , , Let A, B, and C be the projection points of A, B, and C in the image coordinate system; The point where line L1 and ray P1 intersect; Let D1 be the projection point in the image coordinate system; According to the cross-ratio projective invariance, we can obtain ; Thus obtain .

6. The path planning method for cleaning edge defects at the corner of a continuously cast billet according to claim 1, characterized in that, Step S3 includes: S31. When the moving camera (8) moves to the two fixed cameras (02), first calculate the external parameter matrix between the moving camera (8) and one of the fixed cameras (02). Then, calculate the external parameter matrix between the moving camera (8) and the fixed camera (02) on the other side. Finally, the external parameter matrix between the two fixed cameras (02) is obtained: ; S32, Move the mobile camera (8) and obtain the external parameter matrix between all adjacent fixed cameras in sequence based on S31; After conversion, the external parameter matrix of the remaining fixed cameras (02) to the reference camera is obtained. ; S33. Finally, minimize the system reprojection error to obtain the external parameter matrix. Optimize.

7. The path planning method for cleaning edge defects at the corner of a continuously cast billet according to claim 6, characterized in that, Step S31 includes: Using a first-type concentric circular target plane, the external parameter matrices of the target are obtained from the world coordinate system (i.e., the coordinate system established on the first calibration plate) to the fixed camera coordinate system on the left. , and the extrinsic parameter matrix [R] to the moving camera coordinate system m |t m ]; The coordinates of the center point of the first type of concentric circular target in the world coordinate system are: The coordinates of the fixed camera (02) on the left are represented as follows: The coordinates of the moving camera (8) in the coordinate system are represented as P. mi , expressed as a formula After transformation, we can obtain That is, the external parameter matrix of the moving camera (8) to the fixed camera (02) on the left is as follows: ; Similarly, the external parameter matrix from the fixed camera (02) to the moving camera (8) on the right side can be obtained as follows: ; set up For a point in the coordinate system of the fixed camera (02) on the right, its coordinates in the coordinate system of the moving camera (8) can be expressed as follows: Then its coordinates in the coordinate system of the fixed camera (02) on the left are... The following formula can be used Solve Solving the system of equations simultaneously, we get P. l =(R ml R rm P r +(R) ml t rm +t ml ) The external parameter matrix [R] from the coordinate system of the right fixed camera (02) to the coordinate system of the left fixed camera (02) can be obtained. rl |t rl ] 。 8. The path planning method for cleaning edge defects at the corner of a continuously cast billet according to claim 1, characterized in that, S4 includes: establishing a base coordinate system based on the zero position of the guide rail. Coordinate system; A second calibration plate is mounted on the nozzle (07), and a second type of concentric target is coated on the second calibration plate, with its initial position being... Displace nozzle (07) to position ; in position At this point, the transformation relationship between the center point of the second type of concentric circular target from the reference camera coordinate system to the end effector coordinate system is given by formula V. This indicates the transformation relationship from the end-effector coordinate system to the base coordinate system; This indicates the transformation relationship from the second calibration plate coordinate system to the end execution coordinate system; This indicates the transformation relationship from the second calibration plate coordinate system to the reference camera coordinate system; This indicates the transformation relationship from the reference camera coordinate system to the base coordinate system; Since the relative relationship between the second calibration plate coordinate system and the end effector coordinate system remains constant, formula VI can be obtained. From formulas V and VI, we can obtain: Right now Since the relative position between the reference camera coordinate system and the base coordinate system remains fixed, let's assume... Finally obtained A can be obtained directly from the motion parameters of the nozzle (07) relative to the base coordinate system, B can be obtained from the pose of the second calibration plate plane by positioning the reference camera coordinate system, and X is the external parameter matrix from the reference camera coordinate system to the base coordinate system to be determined.

Images