A method for calculating the conversion relationship between camera coordinate system and laser coordinate system

Abstract

The present application provides a method for calculating the conversion relationship between a camera coordinate system and a laser coordinate system, comprising: obtaining a calibration film image, the calibration film image including an image of a rotation mirror calibration figure and an image of a calibration point. Determine the rotation mirror parameters based on the image of the rotation mirror calibration figure. Correct the calibration film image based on the rotation mirror parameters to generate a corrected calibration film image; extract the target calibration point based on the corrected calibration film image. Obtain the image coordinates and laser coordinates of the target calibration point to calculate the coordinate system conversion relationship. The present application can determine the rotation mirror parameters from the image of the rotation mirror calibration figure obtained by the camera, correct the calibration film image based on the rotation mirror parameters, so that the corrected calibration film image completely corresponds to the calibration film of the laser marking point, so that the image coordinates of the calibration point corresponding to the laser coordinates can be accurately extracted based on the calibration film image, thereby improving the accuracy of camera calibration.

Description

Technical Field

[0001] The present application relates to the field of industrial vision technology, and in particular to a method for calculating the transformation relationship between a camera coordinate system and a laser coordinate system. Background Art

[0002] In the field of industrial vision, camera calibration is a crucial prerequisite for processes such as inspection, measurement, and assembly. Through camera calibration, the camera's internal and external parameters can be calculated, and the relationship between the camera coordinate system and the world coordinate system can be established. This allows for measurement and inspection of product dimensions, defects, and position, enabling automated production. Lasers are widely used in cutting, welding, and marking applications. Lasers emit a laser beam through a laser beam generator. By adjusting the laser beam's emission position, the laser beam can be moved within its operating range to accomplish tasks such as cutting, welding, and marking.

[0003] To ensure the camera can accurately guide the laser for high-precision work, efficient, fast, and accurate conversion of the coordinate systems between the laser and camera is required to achieve camera calibration. Taking advantage of the ease of laser marking, a commonly used calibration method in the industry is to place a calibration sheet within the camera's field of view and secure it. The laser then uses the sheet to mark calibration points, which are then used to calibrate the camera.

[0004] However, when extracting calibration points from the calibration film, cameras at different locations will capture different images of the same calibration film due to different camera installation positions and angles. If the calibration film image captured by the camera is mirrored or rotated relative to the calibration film where the laser marks the calibration points, the image coordinates of the calibration points corresponding to the laser coordinates cannot be accurately extracted from the calibration film image, resulting in reduced camera calibration accuracy. Summary of the Invention

[0005] The present application provides a method for calculating the conversion relationship between the camera coordinate system and the laser coordinate system to solve the problem that the existing camera calibration method cannot accurately extract the image coordinates of the calibration points corresponding to the laser coordinates based on the calibration image.

[0006] This application provides a method for calculating the conversion relationship between a camera coordinate system and a laser coordinate system, including:

[0007] A calibration image is acquired, where the calibration image includes an image of the rotated mirror calibration figure and an image of the calibration points.

[0008] The rotation mirror parameters are determined according to the image of the rotation mirror calibration figure.

[0009] Correcting the calibration film image according to the rotation mirror parameters to generate a corrected calibration film image;

[0010] Target calibration points are extracted according to the corrected calibration image.

[0011] The image coordinates and laser coordinates of the target calibration point are obtained.

[0012] A coordinate system conversion relationship is calculated according to the image coordinates and the laser coordinates.

[0013] In a possible implementation, the rotation mirror calibration figure is L-shaped, and the step of determining the rotation mirror parameters according to the image of the rotation mirror calibration figure includes:

[0014] Obtain a set of line segment vectors in the calibration image.

[0015] L-edge features are extracted according to the line segment vector set, where the L-edge features include: a long-edge vector and a short-edge vector.

[0016] It is determined whether the calibration image has a mirror image according to the long side vector and the short side vector.

[0017] If the calibration image does not have a mirror image, the image rotation parameter is calculated according to the long side vector.

[0018] In a possible implementation, before the step of calculating the image rotation parameter according to the long side vector, the step further includes:

[0019] The long side vector is corrected to obtain an accurate long side vector.

[0020] The image rotation parameter is calculated according to the precise long side vector.

[0021] In a possible implementation, after the step of determining whether the calibration image has a mirror image according to the long side vector and the short side vector, the step further includes:

[0022] If a mirror image exists in the calibration image, the mirror image parameters are calculated according to the long side vector and the short side vector.

[0023] A mirror long side vector is generated according to the mirror parameter, and an image rotation parameter is calculated according to the mirror long side vector.

[0024] In a possible implementation, before the step of calculating the image rotation parameter according to the mirror long side vector, the step further includes:

[0025] Correct the long side vector of the mirror image to obtain an accurate long side vector of the mirror image.

[0026] The image rotation parameter is calculated according to the precise mirror image long side vector.

[0027] In a possible implementation, the target calibration point is circular, and the step of extracting the target calibration point according to the calibrated calibration image includes:

[0028] Obtaining a grayscale adaptive threshold of the calibration image.

[0029] A plurality of regions of the calibration image are extracted according to the grayscale adaptive threshold, each of the regions having the same color.

[0030] The circularity value of the area is calculated.

[0031] A target area is acquired according to the roundness value, where the target area is the area with the highest roundness value.

[0032] A target color is acquired according to the target area, where the target color is the color of the target area.

[0033] The target calibration point is extracted according to the target color.

[0034] In a possible implementation, the step of extracting the target calibration point according to the target color includes:

[0035] A target color region set is acquired according to the target color, where the target color region set includes regions having the target color.

[0036] The roundness value of each area in the target color area set is obtained to generate a target roundness value set.

[0037] A roundness value threshold range is calculated according to the target roundness value set.

[0038] According to the roundness value threshold range and the target color area set, a target color area that meets the roundness value threshold range is obtained as the target calibration point.

[0039] In a possible implementation, after the step of extracting the target calibration points according to the calibrated calibration film image, the step further includes:

[0040] The number and theoretical value of the target calibration points are obtained.

[0041] If the number of the target calibration points is less than the theoretical value, a calibration point matrix is ​​constructed according to the target calibration points.

[0042] The target calibration point matrix is ​​filled to generate a filled calibration point matrix.

[0043] Obtain the image coordinates of the calibration points in the filled calibration point matrix.

[0044] In a possible implementation, the step of constructing a calibration point matrix according to the target calibration points includes:

[0045] Obtain the minimum X-axis coordinate value, the minimum Y-axis coordinate value, the minimum X-axis spacing, the minimum Y-axis spacing, the theoretical number of rows, and the theoretical number of columns of the target calibration point.

[0046] Taking the minimum X-axis coordinate value and the minimum Y-axis coordinate value as the starting point, the minimum X-axis spacing as the X-axis spacing, and the minimum Y-axis spacing as the Y-axis spacing, a calibration point matrix is ​​constructed with the theoretical number of rows as the number of rows and the theoretical number of columns as the number of columns.

[0047] In a possible implementation, the step of filling the target calibration point matrix to generate a filled calibration point matrix includes:

[0048] Traverse each point of the target calibration point matrix in turn to determine whether there is a calibration point at the point.

[0049] If the point has no calibrated point, then check whether the point has a marked point to be filled.

[0050] If the to-be-filled marking point exists in the searched point position, the to-be-filled marking point is filled to generate a filled marking point.

[0051] A filling calibration point matrix is ​​generated according to the target calibration points and the filling mark points.

[0052] In a possible implementation, after the step of generating a filled calibration point matrix according to the target calibration points and the filled marking points, the step further includes:

[0053] Obtain a first number of rows and a first number of columns, where the first number of rows is the number of rows of the matrix of filled calibration points, and the first number of columns is the number of columns of the matrix of filled calibration points.

[0054] It is determined whether the first number of rows is equal to the theoretical number of rows, and whether the first number of columns is equal to the theoretical number of columns.

[0055] If the first number of rows and the theoretical number of rows are not equal and / or the first number of columns and the theoretical number of columns are not equal, then obtain the first row number difference and the first row number difference, where the first row number difference is the difference between the first row number and the theoretical number of rows, and the first row number difference is the difference between the first column number and the theoretical number of columns.

[0056] A reverse filling starting point is determined according to the first row number difference and the first column number difference.

[0057] A reverse filling calibration point matrix is ​​constructed according to the reverse filling starting point.

[0058] In a possible implementation, after the step of constructing a reverse filling calibration point matrix according to the row number difference and the column number difference, the following steps are included:

[0059] Obtain a second number of rows and a second number of columns, where the second number of rows is the number of rows of the reverse-fill calibration point matrix, and the second number of columns is the number of columns of the reverse-fill calibration point matrix.

[0060] It is determined whether the second number of rows is equal to the theoretical number of rows, and whether the second number of columns is equal to the theoretical number of columns.

[0061] If the second row number is not equal to the theoretical row number and / or the second column number is not equal to the theoretical column number, obtain the second row number difference and the second row number difference, the second row number difference is the difference between the second row number and the theoretical row number, and the second row number difference is the difference between the second column number and the theoretical column number.

[0062] The reconstructed X-axis minimum spacing and the reconstructed Y-axis minimum spacing are determined according to the second row number difference and the second column number difference.

[0063] A reconstructed filling calibration point matrix is ​​constructed according to the reconstructed X-axis minimum spacing and the reconstructed Y-axis minimum spacing.

[0064] It can be seen from the above technical solution that the present application provides a method for calculating the conversion relationship between the camera coordinate system and the laser coordinate system, including: obtaining a calibration piece image, the calibration piece image including an image of a rotating mirror calibration figure and an image of a calibration point. Determine the rotational mirror parameters based on the image of the rotating mirror calibration figure. Correct the calibration piece image according to the rotational mirror parameters to generate a corrected calibration piece image; extract the target calibration point according to the corrected calibration piece image. Obtain the image coordinates and laser coordinates of the target calibration point. Calculate the coordinate system conversion relationship based on the image coordinates and the laser coordinates. The present application sets a rotating mirror calibration figure on the calibration piece, and the rotational mirror parameters can be determined by the image of the rotating mirror calibration figure obtained by the camera. The calibration piece image is corrected according to the rotational mirror parameters so that the corrected calibration piece image completely corresponds to the calibration piece of the laser marking point, so that the image coordinates of the calibration point corresponding to the laser coordinates can be accurately extracted based on the calibration piece image, thereby improving the accuracy of camera calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0066] Figure 1 A schematic diagram of the working principle of a galvanometer laser provided in an embodiment of the present application;

[0067] Figure 2 A schematic diagram of a calibration sheet provided in an embodiment of the present application;

[0068] Figure 3 A schematic diagram of a calibration image obtained by cameras at different positions according to an embodiment of the present application;

[0069] Figure 4 A flowchart of a method for calculating the conversion relationship between a camera coordinate system and a laser coordinate system provided in an embodiment of the present application;

[0070] Figure 5 A flowchart of a method for determining rotation mirror parameters provided in an embodiment of the present application;

[0071] Figure 6 A flowchart of a method for extracting target calibration points based on a calibrated calibration film image provided in an embodiment of the present application;

[0072] Figure 7 A schematic diagram of an image blob analysis result provided in an embodiment of the present application;

[0073] Figure 8 A schematic diagram of the conversion relationship between a camera coordinate system and a laser coordinate system provided in an embodiment of the present application;

[0074] Figure 9 A schematic diagram of a poor-quality calibration image provided in an embodiment of the present application;

[0075] Figure 10 A flow chart of a method for constructing a matrix of filled calibration points provided in an embodiment of the present application;

[0076] Figure 11 A schematic diagram of a target calibration point matrix provided in an embodiment of the present application;

[0077] Figure 12 A schematic diagram of constructing a reverse filling calibration point matrix provided in an embodiment of the present application;

[0078] Figure 13 A flow chart of a method for constructing a reverse filling calibration point matrix provided in an embodiment of the present application;

[0079] Figure 14 A schematic diagram of constructing and reconstructing a matrix of filled calibration points provided in an embodiment of the present application;

[0080] Figure 15A flowchart of a method for constructing and reconstructing a matrix of filled calibration points provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0082] In the field of industrial vision, camera calibration is a crucial prerequisite for inspection, measurement, assembly, and other processes. Through camera calibration, the camera's internal and external parameters can be calculated, and the relationship between the camera coordinate system and the world coordinate system can be established. This allows for measurement and inspection of product dimensions, defects, and positions, enabling automated production.

[0083] Laser is a widely used device in cutting, welding, marking and other sub-fields. To illustrate the working principle of a common galvanometer laser, please refer to Figure 1 , Figure 1 This is a schematic diagram of the working principle of a galvanometer laser provided in an embodiment of the present application. Figure 1 As shown in the figure, the galvanometer laser consists of a laser beam generating device, a galvanometer connected to an X-axis motion motor, a galvanometer connected to a Y-axis motion motor, and a flat field lens. In actual application, the incident position of the laser beam remains unchanged. By adjusting the position of the X-axis motor and the Y-axis motor, the emission position of the laser beam can be changed, thereby realizing the movement of the laser beam within the working range to complete tasks such as cutting, welding, and marking.

[0084] In order to ensure that the camera can accurately guide the laser to perform high-precision work, it is necessary to efficiently, quickly, and accurately complete the conversion relationship between the laser and camera coordinate systems to achieve camera calibration. In combination with the convenient characteristics of laser marking, the commonly used calibration method in the industry is to place a calibration piece in the camera's field of view. The material and background of the calibration piece can be uniform. Figure 2 , Figure 2 This is a schematic diagram of a calibration sheet provided in an embodiment of the present application. Figure 2 As shown in the figure, the calibration sheet includes 36 calibration points in 6 rows and 6 columns. When using the calibration sheet, it needs to be fixed. The laser will engrave M rows and N columns of marking points on the calibration sheet. As the laser engraves the marking points, it will record the laser coordinate value of each marking point. The camera can be calibrated based on the calibration points.

[0085] However, when extracting calibration points from the calibration film, due to the different camera installation positions and angles, for the same calibration film, the calibration film images collected by cameras at different positions are also different. Figure 3 , Figure 3 This is a schematic diagram of a calibration image obtained by cameras at different positions according to an embodiment of the present application. Figure 3 As shown in the figure, Camera 1 and Camera 2 are in different positions. For the same calibration film, Camera 1 and Camera 2 obtain different calibration film images due to their different positions. If the calibration film image captured by the camera is mirrored or rotated relative to the calibration film on which the laser marks the calibration points, then the image coordinates of the calibration points corresponding to the laser coordinates cannot be accurately extracted from the calibration film image, resulting in reduced camera calibration accuracy.

[0086] The present application provides a method for calculating the conversion relationship between the camera coordinate system and the laser coordinate system to solve the problem that the existing camera calibration method cannot accurately extract the image coordinates of the calibration points corresponding to the laser coordinates based on the calibration image.

[0087] See also Figure 4 , Figure 4 This is a flow chart of a method for calculating the conversion relationship between the camera coordinate system and the laser coordinate system provided in an embodiment of the present application. Figure 4 As shown, the present application provides a method for calculating the conversion relationship between a camera coordinate system and a laser coordinate system, including steps S101-S106:

[0088] S101: Acquire a calibration image, where the calibration image includes an image of a rotated mirror calibration figure and an image of calibration points.

[0089] In this embodiment, a calibration film image taken by a camera is obtained. A rotation mirror calibration figure and a plurality of calibration points are provided in the calibration film of the present application. The rotation mirror calibration figure is used to determine the rotation mirror parameters. The rotation mirror calibration figure is an irregular figure that can distinguish directions, for example, an L-shape, an asymmetric cross, and a DM code. The calibration points are used to calibrate the camera, and generally include a plurality of calibration points to form a calibration point matrix. The calibration points can be circular, annular, rectangular, and cross-shaped.

[0090] S102: Determine rotation mirror parameters according to the image of the rotation mirror calibration figure.

[0091] In this embodiment, the image of the rotating mirror calibration figure is used to automatically determine the rotating mirror parameters for correcting the calibration piece image so that the corrected calibration piece image corresponds to the calibration piece on which the laser hits the calibration point, so as to eliminate the influence of different camera installation positions on the calibration point image coordinates extracted through the calibration piece image.

[0092] In one possible implementation, taking the L-shaped rotational mirror calibration pattern as an example, the principle for determining the rotational mirror parameters based on the image of the rotational mirror calibration pattern is to primarily calculate the rotational mirror parameters by extracting L-edge features. During automatic feature extraction, all line segments in the calibration image are first identified. A check is then performed to determine whether the line segment set is empty. If so, the input image is invalid, and the process terminates immediately. If not, the next step is to select line segments that meet the L-edge features. These line segments are then used to determine whether the image is mirrored and the image rotation angle.

[0093] For details, please refer to Figure 5 , Figure 5 This is a flow chart of a method for determining rotation mirror parameters provided by an embodiment of the present application. Figure 5 As shown, the step of determining the rotation mirror parameters according to the image of the rotation mirror calibration figure includes the following steps S201-S204:

[0094] S201: Obtain a set of line segment vectors in the calibration image.

[0095] In this embodiment, since the rotation mirror calibration figure is L-shaped, which is composed of two line segments, we can first extract all the line segments in the calibration image by defining an automatic feature search method to generate a set of line segment vectors. Then, we can accurately extract the L-shaped rotation mirror calibration figure based on the set of line segment vectors.

[0096] S202: Extracting L-edge features according to the line segment vector set, where the L-edge features include: a long-edge vector and a short-edge vector.

[0097] In this embodiment, the line segments that meet the requirements are screened out according to the line segment vectors, namely the L-edge features: the long side vector and the short side vector. The specific method of extracting the L-edge features according to the line segment vector set can be implemented by the following steps S2021-S2021:

[0098] S2021: Calculate the angle between any two vectors in the line segment vector set, and obtain two vectors whose angles satisfy the angle threshold. Since the angle between the two vectors constituting the L side is 90°, the angle threshold can be set to 80° to 100°. The angle threshold can be set according to the needs and accuracy requirements of the actual application and is not specifically limited in this application.

[0099] S2022: Calculate the intersection of two vectors that meet the angle threshold, and form two new vectors based on the intersection. Specifically, first obtain the point in each vector that is farthest from the intersection, and then construct a new vector based on the farthest point and the intersection. Using the constructed new vector and the original vector, we can further select two L-edge vectors that meet the conditions.

[0100] S2023: Based on the original vector and the new vector, two L-edge vectors that meet the conditions are retained through filtering. For example, the filtering conditions can be that the ratio of the long side to the short side of the new vector to the old vector is between 1.5 and 2.5, and the ratio of the new vector to the old vector is between 0.8 and 1.2. Finally, the two L-edge vectors that meet the above filtering conditions are retained. It should be noted that the retained vectors are the original vectors, not the newly constructed vectors.

[0101] S2024: Sort the two L-side vectors retained in the previous step according to a sorting rule. The sorting rule is determined based on a preset rotation mirror calibration pattern. For example, in the embodiment of the present application, the rotation mirror calibration pattern is an L-shape with an aspect ratio of 2 and an angle of 90°. Therefore, the sorting rule can be the aspect ratio closest to 2 and the angle closest to 90°. This allows the long side vector and the short side vector of the L-side to be extracted based on the set of line segment vectors.

[0102] S203: Determine whether the calibration image has a mirror image according to the long side vector and the short side vector.

[0103] In this embodiment, after obtaining the long side vector and short side vector of side L through the above steps, the presence of a mirror image is determined based on the long side vector and the short side vector. For example, if the long side vector of side L is L1 = (x1, y1) and the short side vector of side L is L2 = (x2, y2), the difference x1y2 - x2y1 is calculated. If x1y2 - x2y1 < 0, the calibration image is mirrored. Otherwise, the calibration image is not mirrored.

[0104] In one possible implementation, the calibration image is obtained by downsampling. First, the downsampling ratio is determined. Then, before determining whether a mirror image exists based on the long side vector and the short side vector, the original size of the L side is restored based on the downsampling ratio to obtain the original long side vector and the original short side vector. Finally, the calibration image is determined to be mirrored based on the original long side vector and the original short side vector. The specific determination method can be found above and is not detailed here.

[0105] S204: If the calibration image does not have a mirror image, calculate image rotation parameters according to the long side vector.

[0106] In this embodiment, if the calibration image does not have a mirror image, there is no need to perform mirror restoration. The image rotation parameter is directly calculated based on the long side vector of the L side. Taking the long side vector of the L side as: L1 = (x1, y1) as an example, the image rotation parameter is the angle between the long side vector and the vector (1, 0). The angle can be calculated by the following formula:

[0107]

[0108] Wherein, θ is the image rotation angle, that is, the image rotation parameter.

[0109] In one possible implementation, if the calibration image is acquired through downsampling, the long side vector needs to be corrected before calculating the image rotation parameters. Specifically, before calculating the image rotation parameters based on the long side vector, the step further includes: correcting the long side vector to obtain a precise long side vector; and calculating the image rotation parameters based on the precise long side vector.

[0110] In this embodiment, if the calibration image is acquired through downsampling, the acquired long side vector of the L side may have a certain error due to the downsampling ratio and may not accurately reflect the long side characteristics of the L side. Therefore, before calculating the image rotation parameters based on the long side vector, the long side vector needs to be corrected. Specifically, a search area in the calibration image can be calculated based on the long side vector with error, and a straight line can be precisely located in the search area to extract the accurate long side vector.

[0111] S205: If a mirror image exists in the calibration image, calculate mirror image parameters according to the long side vector and the short side vector.

[0112] S206: Generate a mirror long side vector according to the mirror parameter, and calculate an image rotation parameter according to the mirror long side vector.

[0113] In this embodiment, if the calibration image has a mirror image, it is necessary to calculate the mirror image parameters. In this application, the mirror image parameters are the rotation angles of the short side of the L side. According to the mirror image parameters, the long side vector is restored to generate the mirror image long side vector. The specific process includes: obtaining the center point coordinates (CenterX, CenterY) of the calibration image, assuming the long side vector is: (x1, y1), then the mirror image long side vector (X, Y) is:

[0114]

[0115] Where X is the horizontal coordinate of the long side vector of the mirror image, and Y is the vertical coordinate of the long side vector of the mirror image.

[0116] The method for calculating the image rotation parameter according to the long side vector of the mirror image may refer to step S204 and will not be described in detail here.

[0117] In one possible implementation, if the calibration image is acquired by downsampling, the mirror long side vector needs to be corrected before calculating the image rotation parameters. Specifically, before calculating the image rotation parameters based on the mirror long side vector, the step further includes: correcting the mirror long side vector to obtain a precise mirror long side vector. The image rotation parameters are calculated based on the precise mirror long side vector.

[0118] In this embodiment, if the calibration image is acquired through downsampling, the acquired mirrored long side vector of the L side has a certain error due to the influence of the downsampling ratio, and cannot accurately reflect the long side characteristics of the L side. Therefore, before calculating the image rotation parameters based on the mirrored long side vector, the mirrored long side vector needs to be corrected. Specifically, the search area in the calibration image can be calculated based on the long side vector with error and the mirror parameters, and a straight line can be accurately located in the search area to extract the accurate long side vector.

[0119] S103: Correcting the calibration image according to the rotation mirror parameters to generate a corrected calibration image.

[0120] In this embodiment, the calibration image is corrected using the rotational mirror parameters calculated in step S102. The rotational mirror parameters include a rotation angle, and the process of correcting the calibration image based on the rotation angle includes rotating the calibration image clockwise by the rotation angle with the center point of the calibration image as the rotation center, thereby obtaining a corrected calibration image. Correcting the calibration image based on the rotational mirror parameters can eliminate the effects of different camera installation positions and improve the accuracy of extracting target calibration points.

[0121] S104: Extracting target calibration points based on the corrected calibration image.

[0122] In this embodiment, by identifying the calibration point pattern in the calibration image, the target calibration point can be extracted. The target calibration point can be used to determine the coordinate system conversion relationship between the image coordinates and the laser coordinate calculation coordinate system.

[0123] For example, take the target calibration point as a circle, see Figure 6 , Figure 6 This is a flow chart of a method for extracting target calibration points based on a calibration image provided by an embodiment of the present application. Figure 6 As shown, in a possible implementation, the step of extracting the target calibration point according to the calibrated calibration film image includes:

[0124] S301: Obtaining a grayscale adaptive threshold of the calibration image.

[0125] In this embodiment, the grayscale value of each pixel in the calibration image can be analyzed and a dynamic threshold representing the image characteristics can be output as a grayscale adaptive threshold, which is used to extract the region of the calibration image.

[0126] S302: extracting multiple regions of the calibration image according to the grayscale adaptive threshold, each of the regions having the same color.

[0127] For example, the background color of the calibration piece in the embodiment of the present application is black, and the calibration points are white. Therefore, the white area and the black area of ​​the corrected calibration piece image can be extracted by Blob analysis. Blob in computer vision refers to a block in the image, that is, a connected area. Blob analysis is to extract and mark the connected domain of the binary image after foreground / background separation. Each marked Blob represents a foreground target, and then some relevant features of the Blob can be calculated. Its advantage is that through Blob extraction, information about the relevant area, such as color, can be obtained. Please refer to Figure 7 , Figure 7 This is a schematic diagram of an image Blob analysis result provided in an embodiment of the present application. Figure 7 As shown, the image area includes white blobs and black blobs, the color of the white blobs area is white, and the color of the black blobs area is black. In this application, if the grayscale values ​​of all pixels in the same area meet certain requirements, it can be considered that each of the areas has the same color.

[0128] S303: Calculate the roundness value of the area.

[0129] Step S302 only extracts different regions with the same color, but the color of the target calibration point is unknown, so it is impossible to determine which region is the calibration point. In this embodiment, since the target calibration point is circular, the circularity value of each region can be calculated based on the characteristics of the circle. The circularity value is used to represent the parameter of roundness. The color of the target region can be obtained based on the circularity value.

[0130] S304: Acquire a target area according to the roundness value, where the target area is the area with the highest roundness value.

[0131] The area with the highest roundness value is obtained as the target area. If the roundness value of this area is the highest, that is, it is closest to a circle, then this area can be considered as the area of ​​the target calibration point.

[0132] S305: Acquire a target color according to the target area, where the target color is the color of the target area.

[0133] The color of the target area is obtained as the target color, and all areas with the target color are likely to be target calibration points. Therefore, the target color is first obtained, and all areas with the target color are extracted according to the target color, so as to further extract the target calibration points.

[0134] S306: Extracting the target calibration point according to the target color.

[0135] In a possible implementation, the step of extracting the target calibration point according to the target color includes:

[0136] S3061: Acquire a target color region set according to the target color, where the target color region set includes regions having the target color.

[0137] S3062: Obtain the roundness value of each area in the target color area set to generate a target roundness value set.

[0138] S3063: Calculate a roundness value threshold range according to the target roundness value set.

[0139] For example, the method for determining the roundness value threshold range includes: sorting the roundness values ​​in the target roundness value set from high to low; calculating the average roundness value of the first half of the roundness values ​​as the reference roundness value, i.e., 1dd; and setting the roundness value threshold range to (0.8dd, 1.6dd). The specific values ​​of the roundness value threshold range can be determined based on actual application and are not specifically limited in this application.

[0140] S3064: According to the roundness value threshold range and the target color area set, obtain a target color area that meets the roundness value threshold range as the target calibration point.

[0141] S105: Acquire the image coordinates and laser coordinates of the target calibration point.

[0142] In this embodiment, the image coordinates of the target calibration point are the extracted center coordinates of the target calibration point, and the laser coordinates of the target calibration point can be directly obtained through laser-related files.

[0143] S106: Calculating a coordinate system conversion relationship according to the image coordinates and the laser coordinates.

[0144] In this example, see Figure 8 , Figure 8 This is a schematic diagram of the conversion relationship between a camera coordinate system and a laser coordinate system provided in an embodiment of the present application. Figure 8As shown in the figure, there is a certain rotation, scaling, and translation relationship between the camera coordinate system and the laser coordinate system. The conversion relationship between the camera coordinate system and the laser coordinate system can be determined by calibration points. The expression of the coordinate system conversion relationship is:

[0145]

[0146] Among them, (Px, Py) is the laser coordinate, indicating the position of the calibration point in the laser coordinate system; (Ix, Iy) is the image coordinate, indicating the position of the calibration point in the image coordinate system; (Tx, Ty) is the laser coordinate of the origin of the image coordinate system, indicating the translation relationship; θ is the angle from the image coordinate system to the laser coordinate system, indicating the rotation relationship, defined as the angle of the positive direction of the X-axis of the image coordinate system in the platform coordinate system; (Sx, Sy) is the actual physical distance of a pixel in the image coordinate system in the X and Y directions, indicating the scaling relationship; Ey indicates the laser coordinate system type. When the laser coordinate system is a left-handed coordinate system, the Ey value is 1, and when the laser coordinate system is a right-handed coordinate system, the Ey value is -1.

[0147] It can be seen from the above technical solution that the present application provides a method for calculating the conversion relationship between the camera coordinate system and the laser coordinate system, including: obtaining a calibration piece image, the calibration piece image including an image of a rotating mirror calibration figure and an image of a calibration point. Determine the rotational mirror parameters based on the image of the rotating mirror calibration figure. Correct the calibration piece image according to the rotational mirror parameters to generate a corrected calibration piece image; extract the target calibration point according to the corrected calibration piece image. Obtain the image coordinates and laser coordinates of the target calibration point. Calculate the coordinate system conversion relationship based on the image coordinates and the laser coordinates. The present application sets a rotating mirror calibration figure on the calibration piece, and the rotational mirror parameters can be determined by the image of the rotating mirror calibration figure obtained by the camera. The calibration piece image is corrected according to the rotational mirror parameters so that the corrected calibration piece image completely corresponds to the calibration piece of the laser marking point, so that the image coordinates of the calibration point corresponding to the laser coordinates can be accurately extracted based on the calibration piece image, thereby improving the accuracy of camera calibration.

[0148] In some application scenarios with complex environments, the quality of the calibration images obtained is poor. Figure 9 , Figure 9 This is a schematic diagram of a poor quality calibration image provided in an embodiment of the present application. Figure 9As shown in the figure, due to the influence of imaging quality, the colors of the calibration points in the calibration image are different. The color of calibration point 1 is white, while the color of calibration point 9 is gray. As a result, it may be impossible to extract all calibration points. If the number of the extracted target calibration points is less than the theoretical number, it means that some target calibration points have not been extracted. Therefore, it is necessary to fill in the missing calibration points to form a complete calibration point matrix and improve the integrity of the calibration point extraction results.

[0149] In one possible implementation, see Figure 10 , Figure 10 This is a flow chart of a method for constructing a matrix of filled calibration points provided in an embodiment of the present application. Figure 10 As shown, in step S104, after the step of extracting the target calibration points according to the calibrated calibration film image, the following steps S401-S404 are also included:

[0150] S401: Acquire the number and theoretical value of the target calibration points.

[0151] The theoretical value of the quantity can be obtained based on the preset calibration piece. Figure 2 Taking the calibration film shown in the figure as an example, the calibration film image consists of a 6-row 6-column calibration point matrix, so the theoretical number of calibration points in the calibration film is 36.

[0152] S402: If the number of the target calibration points is less than the theoretical value, construct a calibration point matrix according to the target calibration points.

[0153] By comparing the number of target calibration points with the theoretical value, it can be determined whether the target calibration points are completely extracted. If the number of target calibration points is less than the theoretical value, it means that there are missing target calibration points, and some target calibration points are omitted and not extracted. In this case, it is necessary to fill in the calibration points to form a complete calibration point matrix.

[0154] In order to find the missing target calibration points accurately and quickly, in one possible implementation, a calibration point matrix is ​​first constructed based on the extracted target calibration points, specifically including the following steps S4021-S4022:

[0155] S4021: Obtain the minimum X-axis coordinate value, the minimum Y-axis coordinate value, the minimum X-axis spacing, the minimum Y-axis spacing, the theoretical number of rows, and the theoretical number of columns of the target calibration point.

[0156] S4022: Using the minimum X-axis coordinate value and the minimum Y-axis coordinate value as starting points, the minimum X-axis spacing as the X-axis spacing, and the minimum Y-axis spacing as the Y-axis spacing, construct a calibration point matrix with the theoretical number of rows as the number of rows and the theoretical number of columns as the number of columns.

[0157] In this embodiment, the calibration point matrix is ​​constructed by taking the theoretical number of rows as 7 and the theoretical number of columns as 7 as an example. Figure 11 , Figure 11 This is a schematic diagram of a target calibration point matrix provided in an embodiment of the present application. Figure 11 As shown, the upper leftmost point of the target calibration point matrix is ​​the starting point, that is, the coordinates of this point are (X-axis minimum coordinate value, Y-axis minimum coordinate value). With this point as the starting point, the X-axis minimum spacing as the X-axis spacing, and the Y-axis minimum spacing as the Y-axis spacing, a calibration point matrix of 7 rows and 7 columns is formed. The theoretical number of the final calibration points of the target calibration point matrix is ​​49.

[0158] S403: Filling the target calibration point matrix to generate a filled calibration point matrix.

[0159] In this embodiment, a search is performed based on the target calibration point matrix to determine whether each point has an extracted target calibration point. If not, the point is filled to generate a filled calibration point matrix.

[0160] In a possible implementation, the step of filling the target calibration point matrix to generate a filled calibration point matrix includes steps S4031-S4034:

[0161] S4031: Traverse each point of the target calibration point matrix in turn to determine whether there is a calibration point at the point.

[0162] S4032: If the point has no calibrated point, then search whether the point has a marked point to be filled.

[0163] In this embodiment, the graphical features of the calibration points can be combined to search for the points to be filled based on these features. For example, if the calibration points are circles, the circle-finding tool can be used to find the points to be filled based on the circle's features. The circle-finding tool can then be used to determine whether it succeeded. If so, the point is marked as a point to be filled; if not, the point is not marked as a point to be filled.

[0164] S4033: If the to-be-filled marking point exists at the searched point position, fill the to-be-filled marking point to generate a filled marking point.

[0165] S4034: Generate a filling calibration point matrix according to the target calibration points and the filling mark points.

[0166] In this embodiment, the target calibration points and the filling mark points are grouped and arranged in rows and columns to generate a filling calibration point matrix.

[0167] S404: Obtain the image coordinates of the calibration points in the filled calibration point matrix.

[0168] In this embodiment, if the extracted target calibration points are less than the theoretical number value, the missing calibration points are filled to form a complete filled calibration point matrix, and the image coordinates of the calibration points are extracted according to the filled calibration point matrix. The image coordinates are used to calculate the coordinate conversion relationship with the laser coordinates.

[0169] In some application scenarios, if the number of rows and columns of the calibration point matrix is ​​still different from the theoretical number of rows and columns, the minimum coordinate values ​​in the X and Y directions calculated based on the extracted target calibration points may not match the actual situation. Figure 12 , Figure 12 This is a schematic diagram of constructing a reverse filling calibration point matrix provided in an embodiment of the present application. Figure 12 As shown in the figure, since the first row and first column are missing, the starting point (Xmin, Ymin) cannot truly reflect the actual situation. In this case, it is necessary to re-determine the starting point based on the difference in the number of rows and columns, and use this starting point as the reverse filling starting point to construct the reverse filling calibration point matrix.

[0170] In one possible implementation, see Figure 13 , Figure 13 This is a flow chart of a method for constructing a reverse filling calibration point matrix provided in an embodiment of the present application. Figure 13 As shown, in step S403, the target calibration point matrix is ​​filled, and the step of generating the filled calibration point matrix further includes steps S501-S505:

[0171] S501: Acquire a first number of rows and a first number of columns, where the first number of rows is the number of rows of the matrix of filled calibration points, and the first number of columns is the number of columns of the matrix of filled calibration points.

[0172] S502: Determine whether the first number of rows is equal to the theoretical number of rows, and whether the first number of columns is equal to the theoretical number of columns.

[0173] S503: If the first number of rows is not equal to the theoretical number of rows and / or the first number of columns is not equal to the theoretical number of columns, obtain a first row number difference and a first row number difference, where the first row number difference is the difference between the first number of rows and the theoretical number of rows, and the first row number difference is the difference between the first number of columns and the theoretical number of columns.

[0174] S504: Determine a reverse filling starting point according to the first row number difference and the first column number difference.

[0175] S505: Constructing a reverse filling calibration point matrix according to the reverse filling starting point.

[0176] In this embodiment, after constructing the reverse fill calibration point matrix, the reverse fill calibration point matrix is ​​filled. Each point in the reverse fill calibration point matrix is ​​traversed again to determine whether the point has a calibration point. If the point does not have a calibration point, it is determined whether to fill the mark point. The specific method is referred to steps S4031-S4033 and is not described in detail here.

[0177] In some application scenarios, if the number of rows and columns of the inverse filled calibration point matrix after filling is still different from the theoretical number of rows and columns, the minimum spacing in the X and Y directions calculated based on the extracted target calibration points may not match the actual situation. Figure 14 , Figure 14 This is a schematic diagram of a method for constructing and reconstructing a matrix of filled calibration points provided in an embodiment of the present application. Figure 14 As shown in the figure, due to the lack of a row in the middle of the calibration point matrix, the minimum spacing in the X and Y directions cannot truly reflect the actual situation. In this case, it is necessary to redefine the minimum spacing in the X and Y directions based on the difference in the number of rows and columns, and rebuild the padded calibration point matrix, which is called reconstructing the padded calibration point matrix.

[0178] In one possible implementation, see Figure 15 , Figure 15 This is a flow chart of a method for constructing and reconstructing a matrix of filled calibration points provided in an embodiment of the present application. Figure 15 As shown, in step S505, after the step of constructing a reverse filling calibration point matrix according to the row number difference and the column number difference, the following steps are included:

[0179] S601: Acquire a second number of rows and a second number of columns, where the second number of rows is the number of rows of the reverse-fill calibration point matrix, and the second number of columns is the number of columns of the reverse-fill calibration point matrix.

[0180] S602: Determine whether the second number of rows is equal to the theoretical number of rows, and whether the second number of columns is equal to the theoretical number of columns.

[0181] S603: If the second number of rows is not equal to the theoretical number of rows and / or the second number of columns is not equal to the theoretical number of columns, obtain a second row number difference and a second row number difference, where the second row number difference is the difference between the second row number and the theoretical number of rows, and the second row number difference is the difference between the second column number and the theoretical number of columns.

[0182] S604: Determine a minimum reconstructed X-axis spacing and a minimum reconstructed Y-axis spacing according to the second row number difference and the second column number difference.

[0183] S605: Constructing a reconstructed filling calibration point matrix according to the reconstructed X-axis minimum spacing and the reconstructed Y-axis minimum spacing.

[0184] In this embodiment, after constructing the reconstructed and filled calibration point matrix, the matrix is ​​filled. Each point in the reconstructed and filled calibration point matrix is ​​traversed again to determine whether it has a calibration point. If the point does not have a calibration point, a determination is made as to whether the marker point should be filled. The specific method is described in steps S4031-S4033 and is not further described here.

[0185] As can be seen from the above technical solutions, this application provides a method for calculating the transformation relationship between the camera coordinate system and the laser coordinate system, and can also automatically extract calibration points based on the calibration film image. If the calibration film image quality is poor and not all calibration points are extracted, a calibration point matrix is ​​constructed and filled in to obtain complete calibration points, thereby improving the completeness and accuracy of the calibration point extraction, thereby improving the accuracy of the calculation of the transformation relationship between the camera coordinate system and the laser coordinate system.

[0186] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without expending creative work shall fall within the scope of protection of this application.

Claims

1. A method for calculating the conversion relationship between a camera coordinate system and a laser coordinate system, characterized in that: include: Acquire a calibration image, wherein the calibration image includes an image of a rotation mirror calibration figure and an image of calibration points; Determining the rotation mirror parameters according to the image of the rotation mirror calibration figure; the rotation mirror calibration figure is L-shaped; Correcting the calibration film image according to the rotation mirror parameters to generate a corrected calibration film image; Extracting target calibration points according to the calibrated calibration film image; Obtaining the image coordinates and laser coordinates of the target calibration point; Calculating a coordinate system conversion relationship according to the image coordinates and the laser coordinates; The step of determining the rotation mirror parameters according to the image of the rotation mirror calibration figure comprises: Obtaining a set of line segment vectors in the calibration image; Extracting L-edge features according to the line segment vector set, wherein the L-edge features include: a long side vector and a short side vector; Determining whether a mirror image exists in the calibration piece image according to the long side vector and the short side vector; If the calibration image does not have a mirror image, the image rotation parameter is calculated according to the long side vector.

2. The method according to claim 1, characterized in that Before the step of calculating the image rotation parameter according to the long side vector, the step further includes: Correcting the long side vector to obtain an accurate long side vector; The image rotation parameter is calculated according to the precise long side vector.

3. The method according to claim 1, characterized in that After the step of determining whether the calibration image has a mirror image according to the long side vector and the short side vector, the following step is further included: If the calibration image has a mirror image, calculating the mirror image parameters according to the long side vector and the short side vector; A mirror long side vector is generated according to the mirror parameter, and an image rotation parameter is calculated according to the mirror long side vector.

4. The method according to claim 3, characterized in that Before the step of calculating the image rotation parameter according to the mirror long side vector, the step further includes: Correcting the long side vector of the mirror image to obtain an accurate long side vector of the mirror image; The image rotation parameter is calculated according to the precise mirror image long side vector.

5. The method according to claim 1, wherein The target calibration point is circular, and the step of extracting the target calibration point according to the calibration image comprises: Obtaining a grayscale adaptive threshold of the calibration image; Extracting a plurality of regions of the calibration image according to the grayscale adaptive threshold, each of the regions having the same color; calculating a circularity value of the area; Acquire a target area according to the roundness value, wherein the target area is the area with the highest roundness value; Acquire a target color according to the target area, where the target color is the color of the target area; The target calibration point is extracted according to the target color.

6. The method according to claim 5, characterized in that The step of extracting the target calibration point according to the target color comprises: Acquire a target color region set according to the target color, wherein the target color region set includes regions having the target color; Obtaining the roundness value of each area in the target color area set to generate a target roundness value set; Calculating a roundness value threshold range according to the target roundness value set; According to the roundness value threshold range and the target color area set, a target color area that meets the roundness value threshold range is obtained as the target calibration point.

7. The method according to any one of claims 1 to 6, characterized in that After the step of extracting the target calibration point according to the calibrated calibration image, the following step is further included: Obtaining the number and theoretical value of the target calibration points; If the number of the target calibration points is less than the theoretical value, constructing a calibration point matrix according to the target calibration points; Filling the target calibration point matrix to generate a filled calibration point matrix; Obtain the image coordinates of the calibration points in the filled calibration point matrix.

8. The method according to claim 7, characterized in that The step of constructing a calibration point matrix according to the target calibration points comprises: Obtain the minimum X-axis coordinate value, the minimum Y-axis coordinate value, the minimum X-axis spacing, the minimum Y-axis spacing, the theoretical number of rows, and the theoretical number of columns of the target calibration point; Taking the minimum X-axis coordinate value and the minimum Y-axis coordinate value as the starting point, the minimum X-axis spacing as the X-axis spacing, and the minimum Y-axis spacing as the Y-axis spacing, a calibration point matrix is ​​constructed with the theoretical number of rows as the number of rows and the theoretical number of columns as the number of columns.

9. The method according to claim 8, characterized in that The step of filling the target calibration point matrix to generate a filled calibration point matrix includes: Traversing each point of the target calibration point matrix in turn to determine whether the point has a calibration point; If there is no calibration point at the point, then check whether there is a mark point to be filled at the point; If the to-be-filled marked point exists at the searched point position, the to-be-filled marked point is filled to generate a filled marked point; A filling calibration point matrix is ​​generated according to the target calibration points and the filling mark points.

10. The method according to claim 7, characterized in that After the step of filling the target calibration point matrix to generate the filled calibration point matrix, the method further includes: Obtain a first number of rows and a first number of columns, where the first number of rows is the number of rows of the matrix of filled calibration points, and the first number of columns is the number of columns of the matrix of filled calibration points; Determining whether the first number of rows is equal to the theoretical number of rows, and whether the first number of columns is equal to the theoretical number of columns; If the first number of rows is not equal to the theoretical number of rows and / or the first number of columns is not equal to the theoretical number of columns, obtaining a first row number difference and a first row number difference, where the first row number difference is the difference between the first number of rows and the theoretical number of rows, and the first row number difference is the difference between the first number of columns and the theoretical number of columns; Determine the reverse filling starting point according to the first row number difference and the first column number difference, A reverse filling calibration point matrix is ​​constructed according to the reverse filling starting point.

11. The method according to claim 10, characterized in that After the step of constructing a reverse filling calibration point matrix according to the row number difference and the column number difference, the following steps are included: Obtain a second number of rows and a second number of columns, where the second number of rows is the number of rows of the reverse-fill calibration point matrix, and the second number of columns is the number of columns of the reverse-fill calibration point matrix; Determining whether the second number of rows is equal to the theoretical number of rows, and whether the second number of columns is equal to the theoretical number of columns; If the second number of rows is not equal to the theoretical number of rows and / or the second number of columns is not equal to the theoretical number of columns, obtaining a second row number difference and a second row number difference, wherein the second row number difference is the difference between the second number of rows and the theoretical number of rows, and the second row number difference is the difference between the second number of columns and the theoretical number of columns; Determine a minimum reconstructed X-axis spacing and a minimum reconstructed Y-axis spacing according to the second row number difference and the second column number difference; A reconstructed filling calibration point matrix is ​​constructed according to the reconstructed X-axis minimum spacing and the reconstructed Y-axis minimum spacing.

Images